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Abstract 


Situation  awareness  (SA)  is  the  psyehologieal  ability  and  capacity  to  perceive 
information  and  act  on  it  acceptably.  This  ability  is  central  to  human  behavior. 

Existing  theoretical  models  explain  many  aspects  of  SA;  however,  knowledge  about  its 
development  out  of  basic  perceptual  abilities  was  insufficient.  This  quantitative  research 
examined  basic  neurocognitive  factors  in  order  to  identify  their  specific  contributions  to 
the  formation  of  SA,  to  address  this  fundamental  discontinuity  in  theory.  Piloting  was  the 
chosen  task.  Visual  attentiveness  (Va),  perceptiveness  (Vp),  and  spatial  working  memory 
(Vswm)  were  assessed  as  predictors  of  SA  under  varying  task  difficulty.  Factorial  and 
repeated-measures  ANOVAs,  Pearson  correlation,  and  linear  multiple  regression 
modeling  were  used  to  determine  the  effects  of  these  independent  variables  on  the 
dependent  variable  SA  and  the  interactions.  The  study  participants  were  19  C-27J  pilots, 
selected  from  the  Ohio  Air  National  Guard.  Neurocognitive  tests  were  administered  to 
the  participants  prior  to  flight.  In-flight  SA  was  objectively  and  subjectively  assessed  for 
24  flights.  At  the  completion  of  this  field  experiment,  the  data  were  analyzed  and  the 
tests  were  statistically  significant  for  the  three  predictor  visual  abilities  Vp,  Va,  and  Vswm 
as  task  difficulty  was  varied,  F(3,l  1)  =  %.\25,p=  .008.  In  addition,  multiple  regression 
analyses  revealed  that  the  visual  abilities  together  predicted  a  majority  of  the  variance  in 
SA,  R  =  0.753,  =  .008.  Moreover,  the  Pearson  correlation  results  indicated  that  Vp 

(r[12]  =  -0.816, =  .002)  had  the  strongest  relationship  of  the  three  neurocognitive 
factors  for  the  overall  flight.  Post-hoc  tests  revealed  a  Cohen’s  =  3.05  yielding 
statistical  power  to  be  0.98.  This  indicates  that  possessing  the  ability  to  have  a 
perceptivity,  to  be  insightful,  and  to  have  discernment,  is  most  important.  During  high 
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task  difficulty  Va  {r\\2\  =  -0.583, =  .046)  had  the  strongest  correlation  with  SA,  while 
during  low  task  diffieulty  it  was  Vswm  (^[12]  =  -0.634, /»  =  .026).  This  work  results  in  a 
significant  contribution  to  the  field  by  providing  an  improved  understanding  of  SA,  an 
Enhanced-Theoretical  Model  of  SA,  and  potentially  safer  travel  for  soeiety  worldwide.  It 
is  reeommended  researeh  be  extended  to  other  populations. 
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Chapter  1:  Introduction 

Cognitive  situation  awareness  has  been  a  topie  of  research  in  the  behavioral 
sciences  for  some  time.  Situation  awareness  may  be  simply  defined  as  the  human  ability 
to  perceive  and  comprehend  the  environment,  and  use  that  information  to  accomplish  a 
task  (Endsley,  1995  a).  Previous  exploration  had  suggested  that  cognitive  situation 
awareness  is  critical  to  a  broad  range  of  human  performance,  but  most  notably  in  aviation 
(Bailey,  Shelton,  &  Arthur,  2011;  Cass,  2011;  Crawford  &  Neal,  2006;  Salmon,  et  ah, 
2010;  Kim,  2009;  Wickens,  2008).  In  aviation  from  1980  to  2007,  983  accidents 
occurred  worldwide  involving  aircraft  without  an  installed  Head  Up  Display  (HUD) 
(Flight  Safety  Foundation,  2009).  A  recent  review  of  this  accident  data  revealed  that  the 
majority  were  due  to  reduced  cognitive  situation  awareness  and  increased  mental 
workload  of  the  aviator,  combined  with  limited  visibility,  and  operations  into  austere 
locations  (Arthur,  Prinzel,  Williams,  &  Kramer,  2006;  Bulkley,  Dyre,  Few,  &  Caufield, 
2009;  Flight  Safety  Foundation,  2009;  NTSB,  2009).  Despite  the  importance  of  situation 
awareness  in  human  performance,  there  was  no  consensus  on  predictors  or  antecedents  (it 
was  unpredictable),  an  area  that  tended  to  be  elusive  and  theoretically  vague  (Cass,  2011; 
Douglas,  Aleva,  &  Havig,  2007;  Dekker  &  Hollnagel,  2004;  Gorman,  Cooke,  &  Winner, 
2006;  Harbour,  Hudson,  &  Zehner,  2012;  Jodlowski,  2008;  Rousseau,  Tremblay,  & 
Breton,  2004;  Sulistyawati,  Wickens,  &  Chui,  2011;  Wickens,  2002,  2008).  A  better 
understanding  of  the  antecedents  of  situation  awareness  will  advance  the  theory  of  how 
situation  awareness  is  formed  and  maintained,  as  well  as  produce  improvements  that 
could  apply  to  any  task  involving  visual  displays  and  information  integration. 
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The  focus  of  this  project  was  to  conduct  neuroergonomic  empirical  studies  that 
would  identify  the  basic  abilities  that  are  essential  antecedents  of  situation  awareness 
(causing  it  to  be  predictable).  As  the  scientific  concept  of  situation  awareness  must  be 
defined  with  respect  to  a  particular  task,  this  project  focused  on  the  complex  multitask  of 
piloting  a  USAF  aircraft.  To  reduce  the  confounds  of  variance  in  pilot  populations  and 
aircraft  types,  the  scope  was  confined  to  the  USAF  Air  National  Guard  population  who 
fly  tactical  airlift  in  north  central  Ohio  with  emphasis  on  task  saturating  phases  of  flight, 
where  situation  awareness  was  especially  stressed  due  to  increased  mental  workload 
(FSF,  2009).  It  should  enhance  behavioral  science  in  this  area  by  shedding  light  on  the 
significance  of  visual  attention  and  visuospatial  working  memory  as  the  antecedents  or 
predictors  of  cognitive  situation  awareness.  In  doing  so,  this  work  will  answer  the 
literature’s  exigent  call  to  advance  cognitive  situation  awareness  theory  (Endsley,  2012; 
Gutzwiller  &  Clegg,  2012;  Harbour  &  Hudson,  2012;  Sulistyawati,  Wickens,  &  Chui, 
2011;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley,  2008),  while  addressing  a  problem 
of  real-world  significance  (Bulkley,  Dyre,  Lew,  &  Canfield,  2009;  Flight  Safety 
Foundation,  2009;  NTSB,  2009;  Wickens  &  McCarley,  2008). 

Background 

The  goal  of  this  research  was  to  address  a  discontinuity  in  situation  awareness 
theory  that  needed  to  be  linked  by  examining  pilot  neurocognitive  factors  as  predictors  of 
situation  awareness  (SA).  Previous  exploration  suggested  that  SA  is  vital  to  pilot 
performance  and  preventing  accidents  (Bailey,  Shelton,  &  Arthur,  2011;  Cass,  2011; 
Crawford  &  Neal,  2006;  Salmon,  et  ah,  2010;  Kim,  2009;  Wickens,  2008);  however, 
there  was  no  consensus  on  predictors  for  SA,  an  area  that  tended  to  be  mysterious  (Cass, 
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2011;  Douglas,  Aleva,  &  Havig,  2007;  Dekker  &  Hollnagel,  2004;  Gorman,  Cooke,  & 
Winner,  2006;  Harbour,  Hudson,  &  Zehner,  2012;  Jodlowski,  2008;  Rousseau,  Tremblay, 
&  Breton,  2004;  Sulistyawati,  Wickens,  &  Chui,  2011;  Wickens,  2002,  2008). 

This  neuroergonomic  study  reports  results  obtained  from  USAF  pilots  performing 
routine  training  missions  (non-simulated).  Cognitive  and  visual  abilities  were  measured 
and  tested  as  predictors  of  SA,  which  were  assessed  by  both  post  flight  subjective  self- 
report  and  in  flight  objectively  measured  changes  in  heart  rate,  heart  rate  variability,  and 
brainwave  activity.  Cognitive  and  visual  ability  were  tested  via  the  Integrated  Visual  and 
Auditory  performance  plus  (IV A+)  test,  which  incorporates  measurements  of  response 
time,  mental  processing  speed,  working  memory,  divided  attention,  response  inhibition, 
comprehension,  accuracy,  and  vigilance  (Brain  Train,  2010;  Corbett  &  Constantine, 

2007;  Tinius,  2003;  Turner  &  Sandford,  1995).  Specifically,  visual  abilities  were 
associated  with  changes  in  SA  and  pilot  mental  workload  in  memory-intensive  flight 
environments,  as  a  function  of  phase  of  flight  and  display  used  (Parasuraman,  Sheridan, 

&  Wickens,  2008;  Parasuraman  &  Wilson,  2008).  As  will  be  discussed  next,  different 
cockpit  displays  or  display  designs  promote  or  impair  SA,  producing  meaningful 
variation  against  which  to  correlate  visual  ability.  The  results  add  to  and  modify  the 
Theoretical  Model  of  Situation  Awareness,  wherein  ability  (in  this  case  visual)  is 
predictive  of  greater  SA. 

Neuroergonomics  is  an  emerging,  interdisciplinary  area  of  research  whose 
purpose  is  to  enhance  knowledge  of  brain  activity,  function,  and  human  behavior  as 
encountered  at  work  and  in  natural  settings  (Parasuraman,  Christensen,  &  Grafton,  2012). 
It  is  integrated  research  between  psychology,  cognitive  neuroscience,  human  factors. 
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engineering,  computer  science,  ergonomics,  and  medicine  (Lees,  Cosman,  Lee,  Fricke,  & 
Rizzo,  2010).  For  example,  neuroergonomics  can  focus  on  the  psychology  of  perceptual 
and  cognitive  functions  and  actions  in  relation  to  actual  technologies  (Parasuraman  et  ah, 
2012).  A  premise  of  this  multidisciplinary  approach  is  that  knowledge  in  the 
psychological  sciences  will  be  enhanced.  Neuroergonomics:  The  Brain  at  Work 
(Parasuraman  and  Rizzo,  2008)  indicated  this  composite  exploration  is  growing  and  is 
being  fueled  by  the  emergence  of  information-saturated  information  display  (ID) 
technology  that  is  now  being  used  by  humans  for  activities  requiring  divided  attention 
and  multitasking  (Parasuraman,  Christensen,  &  Grafton,  2012).  Neuroergonomics 
provides  a  novel  approach  as  a  contemporary  perspective  in  science,  with  human 
situation  awareness  (SA)  research  as  an  integral  part  (Lees,  et  ah,  2010;  Parasuraman  et 
ah,  2012;  Parasuraman  &  Rizzo,  2008;  Parasuraman,  Sheridan,  &  Wickens,  2008; 
Parasuraman  &  Wilson,  2008;  Wilson,  Estepp,  &  Davis,  2009).  This  work  adopted  a 
neuroergonomic  approach,  as  will  be  discussed  in  detail  later. 

In  the  literature,  SA  is  often  discussed  using  philosophical  concepts  (Blandford  & 
Wong,  2004;  Dekker  &  Hollnagel,  2004;  Elliott  et  ah,  2009;  Gorman,  Cooke,  &  Winner 
2006;  Jodlowski,  2008;  Nullmeyer,  Herz,  &  Montijo,  2009;  Rousseau,  Tremblay,  & 
Breton,  2004;  Shelton,  et.  al,  2009).  Situation  Awareness  (SA)  is  currently  viewed  as  a 
complicated  tangible  that  exists  and  necessitates  a  keen  sense  for  visual  cues  (Billingsley, 
Kuchar,  &  Jacobson,  2001;  Endsley,  2012;  Parasuraman,  Sheridan,  &  Wickens,  2008). 

As  a  starting  point,  SA  could  be  described  as  the  cognitive  awareness  level  by  the  pilot  as 
it  relates  to  aircraft  flight  (Tsang  &  Vidulich,  2003,  2006;  Vidulich  &  Tsang,  2012). 
Bringing  into  consciousness  the  characteristics  that  develop  during  tasks  in  order  to 
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project  into  the  future  is  SA  (Endsley,  1995a,  1995b;  Wickens,  1992).  At  the 
fundamental  level  of  SA,  the  pilot  needs  to  accurately  perceive  relevant  information 
(Endsley,  1995a,  1995b).  Endsley’s  (2012)  Theoretical  Model  of  SA  (TMSA)  submits 
there  are  a  total  of  three  levels  for  SA;  a)  perception  (Eevel  1),  b)  comprehension  (Eevel 
2),  and  c)  projection  (Eevel  3). 

The  concept  of  Mental  Workload  (WE)  is  the  level  of  demand  imposed  by  tasks 
on  the  pilot’s  limited  mental  resources  (Wickens  &  Hollands,  2000).  It  is  related  to  both 
demand  and  cognitive  capacity  (Wickens,  2008).  The  demand  is  imposed  by  mental 
tasks  on  the  human,  whether  considered  as  single  or  multiple  (Wickens,  2008).  WE  in 
aviation  is  environment  driven;  therefore,  it  is  driven  by  the  stimulus  and  activity 
involved,  and  it  is  not  something  that  is  necessarily  self-paced.  Abstractly,  WE  can 
reflect  a  subjective  experience  of  mental  effort  (Dekker  &  Hollnagel,  2004).  In 
straightforward  and  simple  terms,  WE  is  the  level  of  mental  effort  a  given  individual  has 
when  performing  tasks  such  as  aviating,  which  consume  cognitive  capacity.  In  summary, 
WE  is  a  function  of  both  task  difficulty  and  individual  capability  (J.C.  Christensen, 
personal  communication,  Jan  10,  2012).  As  one  measures  the  current  psychological  and 
physiological  state  of  an  individual,  one  is  measuring  mental  workload  and  in  turn, 
cognitive  situation  awareness  (Gutzwiller  &  Clegg,  2012). 

Neuroergonomic  methods  provide  a  framework  for  this  field  study  integrating 
subjective  and  objective  measures  using  a  multidisciplinary  approach  to  studying  the 
brain  at  work  in  the  cockpit  measuring  possible  SA  predictors.  Adopting 
neuroergonomic  methods,  psychophysiological  assessment  of  the  effects  of  varying  task 
difficulty  (based  on  display  design)  in  actual  flight  with  neurocognitive  factors  as 
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independent  variables  has  substantial  potential  for  improving  our  understanding  of  SA 
and  WL,  and  the  relationship  between  the  two  (Bailey,  et  al  201 1;  Campbell,  2010; 
Campbell,  Castaneda,  &  Pulos,  2010;  Crawford  &  Neal,  2006;  Douglas,  Aleva,  &  Havig, 
2007;  Flight  Safety  Foundation,  2009;  Geiselman  &  Havig,  2011;  Harbour,  Christensen, 
Estepp,  &  Gray,  2012;  Harbour,  Hudson,  &  Zehner,  2012;  Kim,  2009;  Kramer  et  ah, 
2005;  Wickens,  2008;  Wiekens,  Levinthal,  &  Riee,  2010).  This  study  was  foeused  on  SA 
and  WL,  with  display  design  used  as  a  eommonly  eneountered  faetor  that  ean  affeet  WL 
and  SA;  it  was  performed  in  the  aetual  aireraft  as  the  test  bed  utilizing  qualified  pilots  as 
subjeets. 

Deficient  situation  awareness  can  lead  to  fatal  accidents,  in  particular  the  number- 
one  killer  in  commercial  aviation,  controlled  flight  into  terrain  (CLIT,  Wickens  & 
McCarley,  2008).  It  has  been  reported  that  if  situation  awareness  could  be  improved  this 
accident  potential  could  be  reduced  from  33  to  73  percent,  saving  lives  and  an  estimated 
$1  million  per  commercial  and  military  aircraft  over  the  next  10  years  (LSL,  2009; 

Prinzel  &  Risser,  2004;  Rockwell  Collins,  2000).  This  affects  society  worldwide.  This 
dissertation  was  virtuous  in  inquiry  and  discovery,  uncovering  and  determining  the 
scientific  need  and  solution. 

Statement  of  the  Problem 

The  problem  addressed  was  that  despite  the  scientific  exigency,  there  were  no 
precise  predictive  components  of  current  theories  of  situation  awareness  and  associated 
specific  quantifiable  cognitive  and  perceptual  processes  (Douglas,  Aleva,  &  Havig,  2007; 
Ellis  &  Levy,  2009;  Elliott  et  ah,  2009;  Endsley,  2012;  Gillan  et  al.,  2009;  Gugerty,  in 
press;  Gutzwiller  &  Clegg,  2012;  Harbour  et  al.,  2012;  Jen-li,  Ruey-Yun,  &  Ching-Jung, 
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2013;  Jones,  Connors,  &  Endsley,  2011;  Lau,  Jamieson,  &  Skranning,  2013;  Proetor  & 
Vu,  2010;  Sulistyawati,  Wickens,  &  Chui,  2011;  Vidulieh  &  Tsang,  2012;  Wickens, 

2008;  Wiekens  &  McCarley,  2008;  Tirre  &  Gugerty,  1999,  2000).  In  the  face  of  nearly 
two  decades  of  work  since  Endsley’ s  Theoretical  Model  of  Situation  Awareness  (TMSA, 
1995a)  was  first  published,  models  of  situation  awareness  were  still  conceptual  models 
that  provided  little  specificity  with  regards  to  the  neurocognitive  processes  that  are 
necessary  for  the  formation  and  maintenance  of  situation  awareness  (Gillan  et  ah,  2009; 
Eau,  Jamieson,  &  Skranning,  2013;  Vidulieh  &  Tsang,  2012;  Wickens  &  McCarley, 
2008).  While  the  TMSA  recognizes  perception  as  a  critical  first  step,  there  were  no 
specific  or  quantitative  links  between  perceptual  abilities  and  situation  awareness;  nor  has 
subsequent  work  been  able  to  clarify  the  issue,  e.g.  the  effects  of  visual  ability  on  the 
level  of  situation  awareness  (Jen-li,  Ruey-Yun,  &  Ching-Jung,  2013;  Jones,  Connors,  & 
Endsley,  2011;  Vidulieh  &  Tsang,  2012;  Sulistyawati,  Wickens,  &  Chui,  2011).  Needed 
theoretical  advancement  in  this  area  continued  to  be  hampered  by  a  lack  of  specific, 
testable  predictions  regarding  plausible  component  processes;  there  had  been  little 
theoretical  progress  due  to  this  (Douglas,  Aleva,  &  Havig,  2007;  Endsley,  2012; 
Gutzwiller  &  Clegg,  2012;  Harbour  et  ah,  2012;  Vidulieh  &  Tsang,  2012;  Wickens, 

2008;  Sulistyawati,  Wickens,  &  Chui,  2011). 

The  seminal  research  done  by  Tirre  and  Gugerty  (1999,  2000),  Wickens  and 
McCarley  (2008),  Elliott  et  ah,  (2009),  Jen-li,  et  ah,  (2013),  and  Gugerty,  (in  press)  found 
that  visual  processing  (Proctor  &  Vu,  2010),  was  involved  in  situation  awareness 
(Vidulieh  &  Tsang,  2012).  However,  the  specific  component  processes  had  not  been 
explored  in  detail  (Gutzwiller  &  Clegg,  2012;  Gugerty,  in  press;  Harbour  et  ah,  2012; 
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Jen-li  et  al.,  2013;  Jones  &  Endsley,  2012;  Sulistyawati  et  al.,  2011;  Vidulich  &  Tsang, 
2012).  The  current  study  therefore  tested  explicit  hypotheses  regarding  specific  abilities 
that  may  contribute  to  situation  awareness;  the  results  fill  in  a  critical  gap  in  the  TMSA, 
and  in  so  doing  enable  both  theoretical  refinement  (providing  an  Enhanced-Theoretical 
Model  of  SA)  and  practical  applications  such  as  improved  procedures,  training  for  pilots, 
and  display  design  that  improve  flight  safety  (Endsley,  2012;  FSE,  2009;  Prinzel  & 
Risser,  2004;  Rockwell  Collins,  2000;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley, 
2008). 

Purpose  of  the  Study 

Visual  processing  can  be  categorized  as  static  and  dynamic  (Proctor  &  Vu,  2010), 
and  may  be  operationalized  as  visual  attentiveness  and  perceptiveness,  integrated  with 
visuospatial  memory  (Brain  Train,  2010;  Christensen  et  ah,  2013;  Corbett  &  Constantine, 
2007;  Endsley,  2012;  Gugerty,  in  press;  Sandford,  Fine,  &  Goldman,  1995;  Sandford  & 
Turner,  1994;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley,  2008).  Consequently,  the 
primary  purpose  of  this  quasi-experimental  quantitative  study  was  to  test  the  predictive 
value  of  these  specific  candidate  visuo-cognitive  abilities:  (a)  Visual  Attentiveness, 
consisting  of  Vigilance,  Focus,  and  Speed  (b)  Visual  Perceptiveness,  consisting  of 
Prudence,  Consistency,  and  Stamina,  and  (c)  Visuospatial  Working  Memory,  consisting 
of  working  memory  that  is  stored  in  the  visuospatial  sketchpad  of  the  mind  as  predictors 
of  situation  awareness,  e.g.  the  effects  of  visual  ability  on  the  level  of  situation 
awareness.  Therefore,  this  project  tested  the  predictive  value  of  these  variables  as 
factors  between  a  particular  task  to  be  performed  and  the  eventual  outcome  of  situation 
awareness.  As  Pavlov,  Watson,  and  Skinner  conducted  field  experiments  in  order  to 
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contribute  to  psychological  theory  (Cervone  &  Pervin,  2007),  a  field  experiment  in  the 
paradigm  of  neuroergonomics  (Parasuraman,  Christensen,  &  Grafton,  2012)  employing 
quasi-experimental  repeated-measures  (within-partieipants),  was  utilized  for  this  study. 

This  was  the  most  effeetive  and  realistic  method  to  perform  the  research  that 
addressed  the  problem  and  the  purpose.  As  computed  by  a  priori  power  analysis,  this 
study  required  a  sample  size  of  a  minimum  of  12  United  States  Air  Force  (USAF)  pilots. 
These  pilots  were  reeruited  at  an  airport  in  north  eentral  Ohio  by  utilizing  a  recruitment 
memo  and  posters  in  squadron  buildings.  All  participants  were  pilots  qualified  in  the 
aircraft.  Two  display  conditions  varied  the  diffieulty  of  maintaining  SA  by  ehanging  the 
presence  and  position  of  the  flight  information  display.  The  first  condition  had  an 
Information  Display  (ID)  design  that  was  near  optimal  and  eentered  on  the  pilot’s  field  of 
view  and  was  no  more  than  five  degrees  above  or  below  the  pilot’s  line  of  sight  (Harbour, 
Hudson,  &  Zehner,  2012).  The  second  condition  had  an  ID  design  that  was  subop timal 
and  well  out  of  pilot  line-of-sight  (a  3  5 -degree  vertieal  drop)  (Harbour  et  al.,  2012)  see 
Figure  3.  This  display  manipulation  indueed  variation  in  task  difficulty,  with  differential 
effeets  on  both  subjective  and  psychophysiologieal  measures  (Capo-Aponte  et  al.,  2009; 
Proctor  &  Vu,  2010).  This  work  was  successful,  and  will  result  in  significant  elaboration 
of  theoretical  models  of  situation  awareness  as  well  as  enabling  and  focusing  efforts  to 
improve  situation  awareness. 

Theoretical  Framework 

The  identified  theoretical  foundations  for  this  study  are  based  primarily  in  the 
Theoretieal  Model  of  Situation  Awareness  (TMSA)  as  deseribed  by  Endsley  (1995a). 
There  are  at  least  three  different  theories  for  situation  awareness  (Stanton,  Chambers,  & 
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Piggott,  2001),  the  TMSA  -  a  three  level  model  that  uses  a  eognitive/information 
processing  approach  (Endsley,  1995a),  the  theory  of  activity  model  that  describes 
situation  awareness  (Bedny  &  Meister,  1999),  and  the  perceptual  cycle  theory  model  that 
is  an  ecological  approach  (Niesser,  1976;  Smith  &  Hancock,  1995).  These  theories 
diverge  in  their  foundational  psychological  construction  (Salmon  et  ah,  2008).  The 
theory  of  activity  model  varies  from  the  TMSA  in  that  it  describes  situation  awareness 
using  eight  functional  blocks  and  posits  that  the  degree  to  which  blocks  are  engaged  is 
contingent  upon  the  character  of  the  task  and  the  individual’s  goals  (Stanton  et  ah,  2001). 
The  perceptual  cycle  theory  model  is  an  alternative  view  of  situation  awareness,  in  that  it 
is  not  dependent  on  the  domain  or  the  individual,  rather  it  is  a  function  amidst  the 
interaction  of  the  domain  with  the  individual  (Stanton  et  ah,  2001).  This  study  focused 
on  the  TMSA,  as  it  is  by  far  the  most  widely  accepted  model  of  situation  awareness 
(Endsley,  2012;  Vidulich  &  Tsang,  2012). 

Endsley’ s  framework  is  grounded  in  hierarchical  levels  of  cognitive  situation 
awareness  and  is  based  on  information  processing.  In  this  theory  as  narrative,  Eevel  1 
situation  awareness  contains  the  perception  and  processing  of  cues  (Endsley,  2012; 
Wickens,  2008;  Wickens  et  ah,  2008).  The  TMSA  simply  states  that  Eevel  1  situation 
awareness  is  achieved  when  cues  are  perceived.  It  makes  no  particular  claims  about 
which  perceptual  abilities  are  critical  to  this  process  or  how  individual  differences  may 
contribute  to  the  relative  ease  of  achieving  Eevel  1 .  Eevel  2  situation  awareness  is  the 
comprehension  of  the  current  situation  by  utilizing  the  information  gained  from  Eevel  1 
(Endsley,  2000a,  1995a;  Wickens,  2008).  Eevel  3  situation  awareness  is  the  utilization  of 
the  situation  model  to  project  and  predict  the  future  state  (Endsley,  1995a,  2000b; 
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Sulistyawati,  Wickens,  &  Chui,  2011;  Wickens,  2009).  Endsley’s  (1995a,  2002a)  view 
of  situation  awareness  is  becoming  utilized  more  and  more  as  the  basis  for  research  in 
areas  such  as  system  displays  (Capo-Aponte  et  ah,  2009;  Eid,  Johnsen,  &  Brun,  2004). 

The  seminal  research  done  by  Tirre  and  Gugerty  (1999,  2000),  Wickens  and 
McCarley  (2008),  Elliott  et  ah,  (2009),  Jen-li,  et  ah,  (2013),  and  Gugerty,  (in  press)  found 
that  visual  processing  (Proctor  &  Vu,  2010),  was  involved  in  situation  awareness 
(Vidulich  &  Tsang,  2012).  However,  the  specific  component  processes  had  not  been 
explored  in  detail  (Gutzwiller  &  Clegg,  2012;  Gugerty,  in  press;  Harbour  et  ah,  2012; 
Jen-li  et  ah,  2013;  Jones  &  Endsley,  2012;  Sulistyawati  et  ah,  2011;  Vidulich  &  Tsang, 
2012).  This  work,  therefore,  tested  explicit  hypotheses  regarding  specific  abilities  that 
contribute  to  situation  awareness;  the  results  of  this  study  fill  in  this  crucial  gap  in  the 
TMSA. 

Visual  processing  can  be  categorized  as  static  and  dynamic  (Proctor  &  Vu,  2010), 
and  may  be  operationalized  as  visual  attentiveness  and  perceptiveness,  integrated  with 
visuospatial  memory  (Brain  Train,  2010;  Christensen  et  ah,  2013;  Corbett  &  Constantine, 
2007;  Endsley,  2012;  Gugerty,  in  press;  Sandford,  Eine,  &  Goldman,  1995;  Sandford  & 
Turner,  1994;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley,  2008).  Accordingly,  the 
primary  objective  of  this  quasi-experimental  quantitative  study  was  to  test  the  predictive 
value  of  these  specific  candidate  visuo-cognitive  abilities.  Visual  Attentiveness,  Visual 
Perceptiveness,  and  Visuospatial  Working  Memory,  as  predictors  of  situation  awareness. 
Therefore,  this  project  tested  the  predictive  value  of  these  variables  as  factors  between  a 
particular  task  to  be  performed  and  the  eventual  outcome  of  situation  awareness. 
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Sixteen  quantitative  experiments  and  fifteen  researeh  artieles  have  illustrated  the 
need  and  attempted  to  discover  the  detailed  links  between  vision,  cognition, 
predictability,  and  situation  awareness.  Over  this  15-year  period,  up  to  and  including 
current  present  day,  the  research  has  gotten  closer,  illustrating  the  continuing  need  to  fill 
this  gap  in  the  TMSA.  Additionally,  three  other  theories  and  or  models  have  links  to  the 
TMSA  and  they  are  Applied  Attention  Theory  (AAT)  (Wickens  &  McCarley,  2008), 
Multiple  Resource  Theory  (MRT)  (Sarter,  2012),  and  the  Salience  Effort  Expectancy 
Value  (SEEV)  Model  (Wickens  &  McCarley,  2008).  Eive  quantitative  experiments  and 
six  research  articles  have  shown  these  to  be  linked  and  influenced  by  the  TMSA; 
consequently,  this  research  could  benefit  those  theories  as  well.  This  dissertation 
identified  critical  variables  underlying  the  formation  of  situation  awareness  as  well  as  the 
relationships  among  these  variables  filling  this  scientific  void  in  theory  (Elliott  et  al, 
2009),  grounding  the  TMSA  in  specific  and  quantifiable  perception  and  cognitive 
processes. 

What  is  known  is  that  situation  awareness  plays  a  vital  role  in  dynamic  decision¬ 
making  environments  (St.  John  &  Smallman,  2008).  However,  further  controversy 
existed  regarding  specific  details  of  TMSA.  Endsley’s  theory  emphasizes  perception  and 
comprehension  of  the  environment  amid  projection  into  the  future;  however,  it  does  not 
contain  enough  granularity  or  accuracy  in  the  area  of  human  perception  (Elliott  et.  ah, 
2009;  Gorman  et  ah,  2006;  Jodlowski,  2008;  Stanton  et  ah,  2001;  Wickens,  2008).  The 
TMSA  does  not  emphasize  reflective  relationships  between  mental  models  and 
knowledge  of  the  present  system. 
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Based  on  multiple  regression  analyses,  dynamic  visual  processing,  visual  search, 
time-sharing,  and  temporal  processing  were  the  attributes  that  made  significant  unique 
contributions  under  the  TMSA  framework,  however  the  exact  knowledge  of  how,  in  what 
way,  and  or  why  they  affected  situation  awareness  was  unknown  (Tirre  &  Gugerty,  1999, 
2000).  In  this  seminal  work  the  other  two  discovered  attributes;  static  visual  processing 
and  working  memory,  were  corollary  but  with  “  mixed  evidence,”  (Tirre  &  Gugerty, 
1999,  p.  18).  The  working  memory  (WM)  and  SA  relationship  results  as  part  of  the 
multivariate  analysis  were  inconclusive  with  little  to  no  correlation,  but  do  play  a  role  in 
some  way,  as  well.  That  work  did  not  clarify  which  basic  visual  processing  elements, 
such  as  visual  attentiveness  and  visual  perceptiveness  (as  characteristics  of  dynamic 
visual  processing)  are  significant  predictors  of  SA. 

Tirre  and  Gugerty  (1999,  2000)  indicated  that  even  though  this  research  opened 
the  door  to  examining  candidate  cognitive  factors  for  situation  awareness,  the 
understanding  of  visual  processing  factors  operating  in  a  dynamic  environment  was 
incomplete,  and  hoped  that  improved  understanding  of  the  role  played  by  these  abilities 
in  forming  and  maintain  situation  awareness  would  be  achieved  in  future  research. 

Wickens  (2008)  clearly  showed  that  pilots  who  failed  to  detect  an  unexpected 
event  lacked  complete  Level  1  situation  awareness;  consequently,  display  design  and 
attentional  tunneling  are  influences  on  situation  awareness.  However,  the  degree  to 
which  improved  visual  attention  prevents  attentional  tunneling,  hence  improving  situation 
awareness,  was  unknown.  Applied  Attention  Theory  (AAT)  (Wickens  &  McCarley, 
2008)  indicates  that  visual  attention  control,  scanning,  information  sampling,  visual 
search,  spatial  attention  and  displays  play  a  role  in  pilot  mental  workload,  which  in  turn 
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would  imply  an  influence  on  situation  awareness  as  well.  Wickens  (2008)  discovered 
that  visual  search  and  attention  were  unique  factors  for  workload.  However,  an 
understanding  of  visual  processing  factors  operating  in  dynamic  environments  related  to 
attention  was  incomplete  (Wickens  &  McCarley,  2008,  p.  38).  This  too  is  linked  to  the 
TMSA. 

This  recent  finding  by  Elliot  et  al.  (2009)  reemphasized  that  vision  is  the  most 
dominant  sense  when  it  comes  to  influencing  situation  awareness  (Wade  &  Swanston, 
2012),  consequently  focusing  research  on  elucidating  the  visual  attributes  for  situation 
awareness  will  have  the  greatest  impact.  Elliot  et  al.  (2009)  provided  additional 
knowledge  to  build  upon  when  researching  visual  cuing,  displays,  and  the  neurocognitive 
factors  involved  in  Eevel  1  situation  awareness  prediction.  This  study  identified  areas  in 
need  of  further  investigation,  specifically  work  on  relating  factors  and  refining  guiding 
principles  to  determine  when,  why,  and  how  cues  come  into  play  to  support  human 
performance  in  demanding  or  complex  environments.  Studies  need  to  further  investigate 
and  refine  theory-driven  predictions  for  workload  and  demands  for  attentional  abilities  to 
include  visual  cuing  (Elliot  et  al,  2009;  Strater,  Riley,  Eaulkner,  Hyatt,  &  Endsley, 

2006). 

Endsley’ s  view  of  situation  awareness  -  the  TMSA  -  is  often  utilized  as  the  basis 
for  research  in  areas  such  as  system  displays  and  military  operations  (Sulistyawati, 
Wickens,  &  Chui,  2011).  Eevel  3  situation  awareness  is  the  utilization  of  the  situation 
model  to  project  and  predict  the  future  state  (Sulistyawati,  Wickens,  &  Chui,  2011). 

What  was  needed  in  future  research  was  to  specifically  investigate  neurocognitive 
characteristics  such  as  visual  attention  and  visual-spatial  working  memory,  and  the 


15 


effects  on  pilot  workload  and  situation  awareness  (Sulistyawati,  Wickens,  &  Chui,  2011). 
There  was  a  lack  and  need  for  additional  psychological  theory  that  is  based  on  the 
neurocognitive  abilities  to  perceive  the  visual  display’s  cues,  predicting  situation 
awareness  and  workload  that  is  Level  1  situation  awareness  (Sulistyawati,  Wickens,  & 
Chui,  2011).  Research  continued  to  struggle  to  find  detailed  links  between  the  cognitive 
demands  on  pilots  and  situation  awareness. 

The  research  performed  by  Jen-li,  Ruey-Yun,  and  Ching-Jung  (2013)  examined 
display  design  for  unmanned  aerial  vehicle  (UAV)  monitoring,  and  its  effects  on  operator 
situation  awareness,  performance,  and  mental  workload  involving  the  TMSA.  The 
operator  in  UAV  flights  has  to  rely  primarily  on  vision  in  this  agent-based  system, 
consequently  the  question  remained  what  role  does  display  design  and  human  visual 
abilities  play  in  the  human-robot  interface  in  order  to  enhance  situation  awareness  (Jones, 
Connors,  &  Endsley,  2011). 

The  results  illustrated  that  compared  to  the  conventional  display,  the  effects  of  a 
situation-augmented  display  on  flight  completion  time  and  abnormality  detection  time 
were  robust  across  different  workloads  but  error  rate  and  perceived  mental  workload 
were  unaffected  by  the  display  type.  With  the  increasing  complexity  of  new  automation 
technology,  there  is  a  significant  challenge  to  researchers  to  better  understand  the 
underlying  cognitive  and  visual  processes  in  order  to  aid  the  control  operator’s  situation 
awareness.  An  important  point  gained  from  this  study  is  that  the  Level  1  SA  from  the 
TMSA  still  presented  problems  for  these  researchers  in  that  they  had  difficulty  in 
assessing  the  interaction  between  operator  visual  abilities  and  display  usability  (Jen-li  et 
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al,  2013).  More  studies  were  needed  to  address  these  unresolved  and  important  issues 
(Jen-li  et  ah,  2013)  expanding  the  TMSA. 

The  very  recent  in-press  work  by  Gugerty  investigated  and  discussed  probable 
component  processes,  both  perceptual  and  cognitive,  that  support  situation  awareness 
during  real-time  tasks.  The  psychological  field  needs  a  better  understanding  for  the 
foundation  of  theoretical  models,  so  that  through  empirical  evidence  a  better 
conception  between  situation  awareness  and  its  component  processes  can  be  achieved 
(Gugerty,  in  press).  Situation  awareness  is  a  complex  process  that  requires  further 
assessment  (Gugerty,  in  press). 

Gugerty  posited  that  the  cognitive  process  for  Level  1  SA  could  potentially  be 
automatic,  and  therefore  would  place  almost  no  demands  on  cognitive  resources;  however, 
this  does  not  explain  the  cognitive  demands  of  attention  and  vigilance,  which  are  known  to 
be  demanding  and  stressful  (Warm,  Parasuraman,  &  Matthews,  2008).  Gugerty  contested 
that  the  TMSA  three-level  view  of  SA  processes  is  at  odds  when  it  comes  to  maintaining 
SA  versus  acquiring  SA.  Future  research  needs  to  narrow  or  broaden  this  view  (Gugerty,  in 
press). 

The  work  of  Gugerty  indicated  that  increasing  SA  knowledge  further,  needs  to  be 
accomplished  by  discovering,  measuring,  and  quantitatively  explaining,  describing,  and 
linking  these  specific  key  perceptual  factors  (visual  abilities),  with  the  goal  being  to 
objectively  fill  the  foundational  gaps  solidly  grounding  the  TMSA.  The  following  relates 
the  TMSA  with  AAT,  MRT,  and  the  SEEV  Model.  Wickens  and  McCarley’s  (2008) 
Applied  Attention  Theory  (AAT),  offers  further  support  for  this  research  in  that  AAT 
indicates  that  visual  attention  control,  scanning,  information  sampling,  visual  search. 
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spatial  attention  and  displays  play  a  role  in  pilot  mental  workload,  which  in  turn  would 
imply  an  influence  on  situation  awareness  as  well,  linking  it  to  the  TMSA.  Additionally, 
further  AAT  clearly  illustrates  that  an  understanding  of  visual  processing  factors 
operating  in  dynamic  environments  related  to  attention  was  incomplete,  providing  further 
evidence  of  the  necessity  of  this  dissertation  study  (Wickens  &  McCarley,  2008). 
Moreover,  Multiple  Resource  Theory  (MRT)  (Bulkley  et.  al,  2009;  Lei  &  Roetting,  2011; 
Pickel  &  Staller,  2012;  Sarter,  2012;  Vidulich  &  Tsang,  2012)  and  the  Salience  Effort 
Expectancy  Value  (SEEV)  model,  which  is  linked  to  MRT  through  the  visual  modality, 
articulates  the  theoretical  foundations  of  this  study. 

MRT  is  evolving  and  feeding  application  models  as  tools  in  improving  interface 
design  (Sarter,  2012).  As  discussed  by  Eei  and  Roetting  (2011),  Wickens  has  made 
advancements  to  MRT  both  in  2002  and  again  in  2008.  These  advancements  have 
expanded  MRT  while  adding  detail  and  granularity.  Current  MRT  submits  that  there  are 
limitations  to  attention  resources  as  before.  Among  other  changes,  current  evidence 
supports  that  two  aspects  of  visual  processing,  referred  to  as  peripheral  and  focal  (foveal) 
vision,  appear  to  sometimes  draw  on  separate  resources,  therefore  tasks  that  draw  on  both 
aspects  may  result  in  relatively  improved  performance,  while  simultaneous  tasks  that  load 
on  only  one  aspect  are  likely  to  degrade  performance  when  these  processes  occur 
simultaneously  (Sarter,  2012),  and  these  have  potential  direct  links  to  the  TMSA. 
Specifically,  Eevel  1  SA  should  be  enhanced  by  distributing  information  across  multiple 
sensory  modalities  as  well  as  between  focal  and  peripheral  vision;  conversely,  SA  should 
be  impaired  by  overloading  one  particular  input  modality.  An  important  characteristic  of 
this  view  of  MRT  is  the  proposal  that  the  effect  of  task  demand  and  resources  may  not  be 
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just  qualitative  but  may  also  involve  quantitative  relationships  (Embrey  et  al.,  2006;  Lei 
&  Roetting,  2011;  Padgett,  2004;  Barter,  2012;  Vidulich  &  Tsang,  2012). 

A  theoretical  model  of  selective  attention  called  the  Salience  Effort  Expectancy 
Value  (SEEV)  model  links  basic  research  psychological  models  such  as  the  TMSA, 
salience  and  attention  capture,  engineering  models,  and  expected  value  optimization 
(Wickens  &  McCarley,  2008).  An  example  of  SEEV’s  purpose  could  be  to  improve  ID 
design  in  order  to  counteract  inattention  (Ma  &  Kaber,  2007;  Matthews,  Bryant,  &  Webb, 
2001). 

The  SEEV  is  linked  to  MRT  primarily  through  visual  modality  and  it  utilizes 
MRT  in  the  creation  of  an  attention  based  applications  tool.  More  specifically,  the  SEEV 
uses  foveal  vision  as  attention  directly  from  the  MRT  and  potentially  the  TMSA.  The 
parameters  of  the  SEEV  (Wickens  &  McCarley,  2008),  which  drive  the  visual  attention 
around  the  environment,  are:  (1)  Salience  (S),  the  exogenous  attention  capturing 
properties  of  events,  e.g.  bright  symbols  that  are  salient  on  the  ID,  (2)  Effort  (Ef)  that 
inhibits  the  movement  of  attention  across  longer  distances,  (3)  Expectancy  (Ex),  the 
probability  of  seeing  an  event  at  a  particular  location,  an  endogenous  cognitive  factor  that 
is  calibrated  by  the  individual  to  the  frequency  of  events  that  occur  at  that  location,  and 
(4)  Value  (V),  which  is  the  importance  (value)  of  tasks  provided  by  the  attended  event,  in 
addition  to  the  relevance  of  the  event  to  a  valued  task. 

With  the  application  of  the  SEEV  model,  aircraft  cockpits  could  be  designed  such 
that  the  location  and  appearance  of  symbology  on  the  display  more  effectively 
corresponds  with  the  actual  environment  in  the  most  efficient  prioritized  manner.  What 
the  SEEV  model  lacks  is  the  influence  of  individual  ability  on  visual  attention  (individual 
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as  “Liveware”;  Edwards,  1972),  i.e.  the  levels  of  the  individual’s  abilities  in  visual 
attentiveness,  visual  pereeptiveness,  and  visual  spatial  working  memory.  Further 
individualization  of  SEEV  would  advanee  the  theory  by  elueidating  the  effects  of 
individual  differences  in  visual  abilities  on  resultant  visual  attention,  which  could 
conceivably  result  in  individually  customizable  information  displays  that  maximize 
usability. 

There  was  a  lack  and  need  for  additional  psychological  theory  that  is  based  on  the 
neurocognitive  abilities  to  perceive  the  visual  display’s  cues,  predicting  situation 
awareness  and  workload  (Harbour,  et.  ah,  2012;  Kim,  2009;  Endsley,  2012;  Sulistyawati, 
et.  ah,  2011;  Vidulich  &  Tsang,  2012)  succinctly  quantitatively  grounding  the  TMSA. 
This  is  covered  and  expanded  upon  further  in  the  Eiterature  Review.  This  Ph.D.  study 
makes  a  solid  contribution  to  the  literature  in  behavioral  science  by  addressing  a  gap  in 
cognitive  situation  awareness  theory  by  way  of  examining  both  pilot  neurocognitive 
factors  and  flight  display  three-dimensional  placement,  and  how  and  why  they  affect 
situation  awareness  and  workload. 

Prior  research  is  incomplete  with  regards  to  the  exact  role  of  visual  processing 
abilities;  future  research  needed  to  refine  measurement  of  these  abilities  and  was 
accomplished.  Seminal  work  performed  by  Crawford  and  Neal  (2006),  and  Gorman, 
Cooke,  and  Winner  (2006)  indicated  the  need  for  additional  research  examining  the 
impact  of  HUD  usage  on  situation  awareness  in  flight,  in  particular,  with  respect  to  the 
prediction  of  situation  awareness.  The  critical  variables  underlying  the  formation  of 
situation  awareness  as  well  as  the  possible  relationships  among  these  variables  needed  to 
be  identified.  One  assumption  of  the  TMSA  is  that  visual  and  cognitive  factors  influence 
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situation  awareness  (Endsley,  1995a,  2012;  Vidulieh  &  Tsang,  2012).  Previous  researeh 
has  indieated  that  complex  tasks  or  difficulties  in  operation  (visual  factors  and  task 
difficulty)  results  in  decreased  situation  awareness  (FAA,  2011;  Flight  Safety 
Foundation,  2009;  Kang,  2008;  Kang,  Yuan,  Fiu  ,  &  Fiu,  2008;  NTSB,  2009;  Wickens, 
2008).  In  this  dissertation  study,  the  visual  factors  are  called  “visual  abilities”  and  vary 
as  a  function  of  the  different  individuals  in  the  naturally  occurring  sample.  In  addition,  a 
cognitive  factor  was  called  “task  difficulty,”  which  was  manipulated  by  the  location  of 
the  display.  According  to  the  TMSA,  it  could  be  postulated  that  increased  visual 
difficulty  and  or  an  increase  in  mental  workload  will  decrease  the  level  of  situation 
awareness,  however,  the  effects  of  visual  ability  on  the  level  of  situation  awareness 
needed  to  be  discovered,  and  were  by  this  study. 

Research  Questions 

This  quantitative  research  utilized  a  quasi-experimental  design  that  made  use  of 
repeated  measures  comparisons,  with  situation  awareness  as  the  dependent  variable.  The 
within-subjects  and  factors  repeated-measures  AN OVA  was  used  along  with  the 
(Pearson)  correlation,  and  linear  multiple  regression  modeling  in  order  to  measure  and 
determine  the  effects  of  the  variables  (independent  /  dependent),  their  interactions  and 
importance,  and  the  strength  of  association  between  variables.  Each  subject  in  the 
experiment  was  exposed  to  all  levels  and  conditions.  As  mental  workload  goes  up, 
cognitive  situation  awareness  goes  down  (Gutzwiller  &  Clegg,  2012;  Kokar  &  Endsley, 
2012),  consequently,  as  one  measures  the  current  psychological  and  physiological  state  of 
an  individual,  one  is  measuring  mental  workload  and  in  turn,  cognitive  situation 
awareness  (Gutzwiller  &  Clegg,  2012).  Previous  work  has  not,  however,  explored  visual 
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abilities  as  predictors  of  situation  awareness  with  factors  between  task  difficulty  and 
situation  awareness.  The  following  research  questions  were  explored  to  address  the 
overarching  question  of  identifying  predictors  of  situation  awareness  under  varying  task 
difficulty. 

In  all  three  below  questions,  the  visual  abilities  were  evaluated  as  predictors 
(factors)  of  situation  awareness.  Task  difficulty,  in  and  of  itself,  was  analyzed  as  a 
necessary  manipulation  check  to  verify  that  the  conditions  produced  significant  main 
effects  on  situation  awareness;  however  this  is  not  a  central  research  question.  A  main 
effect  of  visual  ability  was  probed  with  post  hoc  tests  to  determine  which  abilities 
contributed  to  an  observed  variation  in  situation  awareness,  likewise  any  interaction 
between  visual  abilities  and  task  difficulty  was  assessed  via  post  hoc  tests.  The  bottom 
line  is  what  are  the  effects  of  visual  ability  on  the  level  of  situation  awareness?  A 
detailed  investigation  that  focused  on  the  following  main  research  questions  was 
performed; 

Ql.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  attentiveness  on  situation  awareness? 

Q2.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  perceptiveness  on  situation  awareness? 

Q3.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  spatial  working  memory  on  situation  awareness? 
Hypothesis 

To  address  the  research  questions,  the  following  hypotheses  were  tested: 
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Hlo.  Under  varying  task  difficulty,  there  will  be  no  statistically  significant  effects 
of  inherent  visual  attentiveness,  (as  measured  by  the  Integrated  Visual  and 
Auditory  Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on 
resulting  situation  awareness. 

Hla.  Under  varying  task  difficulty,  there  will  be  statistically  significant  effects  of 
inherent  visual  attentiveness,  (as  measured  by  the  Integrated  Visual  and  Auditory 
Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on  resulting 
situation  awareness. 

H2o.  Under  varying  task  difficulty,  there  will  be  no  statistically  significant  effects 
of  inherent  visual  perceptiveness,  (as  measured  by  the  Integrated  Visual  and 
Auditory  Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on 
resulting  situation  awareness. 

H2a.  Under  varying  task  difficulty,  there  will  be  statistically  significant  effects  of 
inherent  visual  perceptiveness,  (as  measured  by  the  Integrated  Visual  and 
Auditory  Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on 
resulting  situation  awareness. 

H3o.  Under  varying  task  difficulty,  there  will  be  no  statistically  significant  effects 
of  Visuospatial  Working  Memory  Test  results  (AFRL,  2009),  on  resulting 
situation  awareness. 

H3a.  Under  varying  task  difficulty,  there  will  be  statistically  significant  effects  of 
Visuospatial  Working  Memory  Test  results  (AFRL,  2009),  on  resulting  situation 


awareness. 
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Nature  of  the  Study 

This  quasi-experimental  quantitative  study  was  to  test  the  predietive  value  of 
these  speeific,  eandidate  visuo-eognitive  abilities  (visual  attentiveness  and  visual 
pereeptiveness,  and/or  inherent  visuo-spatial  working  memory)  with  respect  to  situation 
awareness  (see  Table  5).  Therefore,  this  project  tested  the  predictive  value  of  these 
variables  as  factors  between  a  particular  task  to  be  performed  and  the  eventual  outcome 
of  situation  awareness.  Additionally,  this  quantitative  study  attempted  to  determine 
whether  using  a  HUD  or  the  HDD  in  the  Cohen’s  Statistical  Power  Analysis  (Cohen, 
1992)  sample  minimum  of  12  pilots,  would  improve  or  worsen  situation  awareness  or 
workload,  and  interact  with  inherent  visual  abilities.  This  study  utilized  a  within-subjects 
design  with  repeated  measures  on  the  two  different  display  conditions  for  each  one  of  the 
subjects.  The  sample  was  from  the  Ohio  Air  National  Guard  (ANG)  pilot  population, 
which  is  relatively  homogeneous. 

This  quantitative  research  employed  the  ANOVA,  along  with  multiple  regression 
analysis  and  the  Pearson  correlation.  The  independent  variables  are  the  inherent  visual 
abilities,  and  the  dependent  variable  is  a  composite  level  of  situation  awareness.  Task 
difficulty  (HUD  and  HDD)  was  varied  across  each  flight  and  was  a  controlled  covariate 
variable. 

Significance  of  the  Study 

This  dissertation  reviewed,  studied,  and  or  critically  analyzed  more  than  300 
references  in  its  construction  and  bore  3-years  to  complete.  Without  research  that  makes 
the  scientific  links  between  basic  visuo-eognitive  abilities  and  situation  awareness,  there 
can  be  little  progress  in  situation  awareness  theory.  Current  models  of  situation 
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awareness  were  still  eonceptual  models  that  provided  low  specifieity  with  regards  to  the 
neurocognitive  processes  that  are  necessary  for  the  formation  and  maintenance  of 
situation  awareness  (Vidulich  &  Tsang,  2012;  Wickens  &  McCarley,  2008). 

This  study  supports  theoretical  advancement  in  this  area  by  providing  evidence 
regarding  plausible  component  processes  (Douglas,  Aleva,  &  Havig,  2007;  Endsley, 
2012;  Gutzwiller  &  Clegg,  2012;  Harbour  et  ah,  2012;  Vidulich  &  Tsang,  2012; 

Wickens,  2008;  Sulistyawati,  Wickens,  &  Chui,  2011).  This  work  tested  explicit 
hypotheses  regarding  these  specific  abilities  that  may  or  may  not  contribute  to  situation 
awareness.  This  effort  was  successful,  and  it  will  result  in  the  problem  that  was 
addressed  being  solved;  the  scientific  exigency  was  treated  with  precise  and  predictive 
metrics  for  current  theories  of  situation  awareness.  The  results  enhance  Endsley’ s 
Theoretical  Model  of  Situation  Awareness  (TMSA,  1995a)  by  providing  specificity 
concerning  the  neurocognitive  processes  that  are  necessary  for  the  formation  and 
maintenance  of  situation  awareness.  This  study  produced  needed  theoretical 
advancement  in  this  area  with  results  regarding  plausible  component  processes; 
generating  theoretical  progress. 

This  study  tested  explicit  hypotheses  regarding  specific  abilities  that  may 
contribute  to  situation  awareness;  the  results  of  this  study  fill  in  a  critical  gap  in  the 
TMSA,  and  in  so  doing  enable  both  theoretical  refinement  and  practical  applications  such 
as  improved  procedures,  training  for  pilots,  and  display  design  that  improve  flight  safety. 
It  will  result  in  significant  elaboration  of  theoretical  models  of  situation  awareness  as 
well  as  enabling  and  focusing  efforts  to  improve  situation  awareness  in  many  tasks  and 
applications.  Deficient  situation  awareness  can  lead  to  fatal  accidents,  in  particular  the 
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number-one  killer  in  commercial  aviation,  controlled  flight  into  terrain  (CFIT,  Wickens 
&  McCarley,  2008).  This  work  is  wholly  successful,  and  results  in  both  a  significant 
addition  to  the  existing  TMSA  and  the  basis  for  developing  techniques  to  better  select 
and  train  pilots,  and  design  displays  reducing  the  incidence  of  fatal  aviation  accidents. 

Definition  of  Key  Terms 

Attention,  It  is  the  act  of  mentally  concentrating  on  a  task  or  tasks  (Matlin, 

2008). 

Attentiveness,  It  consists  of  Vigilance,  Focus,  and  Speed,  and  is  applicable  in 
both  the  static  and  dynamic  environment,  involving  both  static  and  dyamic  visual 
processing  (Brain  Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine,  & 

Goldman,  1995;  Sandford  &  Turner,  1994). 

Awareness,  It  is  the  ability  to  accurately  perceive,  from  both  the  human  senses 
and  perception  in  the  cockpit  about  flight,  present,  past,  and  future  (Matlin,  2008).  In 
real-world  terms  for  aviation,  it  is  the  cognitive  awareness  level  by  the  pilot  as  it  relates 
to  aircraft  flight  (Tsang  &  Vidulich,  2006). 

Binocular  instantaneous  field  of  view  (IFOV),  It  is  the  FOV  visible  to  the  eyes, 
the  left  and  right  at  the  same  time  (Newman,  1995). 

Bore-sight,  It  is  the  information  display  system’s  alignment  with  respect  to  the 
reference  axis  of  the  aircraft  (Newman,  1995). 

Cockpit  Design  Eye  Point  (CDEP),  This  is  the  spatial  location  and  pilot 
position  in  the  cockpit  where  the  pilot  should  sit  in  order  to  properly  operate  the  aircraft 
(Harbour,  Hudson,  &  Zehner,  2011;  Newman,  1995).  This  position  allows  for  optimal 
visibility  both  inside  and  outside,  the  pilot  will  have  optimal  aircraft  controllability  and 
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cockpit  reach,  and  proper  visibility  to  take-off,  fly,  and  land  the  aireraft  safely.  This 
position  is  a  three  dimensional  point  in  spaee  that  will  accommodate  the  pilot  population 
extracted  at  the  mid-pupil. 

Cognition,  It  is  the  human  process  that  is  mental  activity,  which  utilizes  thoughts 
for  aequiring,  proeessing,  storing,  transforming,  and  using  knowledge  (Matlin,  2008). 

Cognitive  Situation  Awareness,  See  Situation  Awareness. 

Combiner,  It  is  the  reflective  sub-system  eomponent  (semitransparent  element  / 
glass  /  lens)  that  is  placed  in  the  pilot’s  forward  FOV  that  provides  and  has  superimposed 
symbology  on  the  external/outside  FOV  of  the  outside  environment  by  utilizing 
eollimation  (Newman,  1995).  It  provides  collimation  by  refleeting  the  light  from  the 
projector  placing  the  light  rays  in  parallel  and  does  not  spread  out  the  light,  as  it 
propagates  the  short  distanee  to  the  pilot’s  eyes  (Wood  &  Howells,  2001).  It  is  designed 
to  project  the  image  source  information  at  optical  infinity.  The  eombiner  has  speeial 
eoatings  that  combined  with  the  lens  simultaneously  reflect  the  HUD  information  and 
transmit  baek,  enabling  the  pilot  to  view  both  the  outside  world  and  the  collimated 
display  (Wood  &  Howells,  2001). 

Consistency,  It  is  the  ability  to  stay  on  task,  respond  reliably,  making  dependable 
responses  in  a  dynamic  environment  (Brain  Train,  2010;  Corbett  &  Constantine,  2007; 
Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner,  1994). 

Divided  attention.  This  is  a  higher  cognitive  process  using  working  memory  to 
perform  two  tasks  near  simultaneously,  or  while  performing  two  cognitively  diverse  tasks 
near  simultaneously  (Wickens  &  McCarley,  2008). 
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Dynamic  Environment,  This  is  a  non-static  or  changing  environment  that 
requires  eontinual  responses  from  the  human  operator.  The  human  operator  is  in  a  state  of 
dis-inhibition  when  working  in  this  environment  (Brain  Train,  2010;  Corbett  & 
Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner,  1994). 

Dynamic  Vision,  It  is  visual  processing  that  occurs  in  the  non-static  or  changing 
environment  (Brain  Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine,  & 
Goldman,  1995;  Sandford  &  Turner,  1994). 

Exit  pupil  (EP),  It  is  a  three  dimensional  disk  in  space  that  contains  all  of  the 
light  collected  by  the  optics  for  the  entire  FOV  (Newman,  1995).  The  maximum  FOV  is 
obtained  at  EP.  The  virtual  aperture  in  an  optical  system  is  the  EP. 

Eye  relief.  It  is  the  distance  from  EP  to  the  FIUD  combiner  surface  (Newman, 

1995) 

Flight  performance.  It  is  how  effectively  the  pilot  aviates,  navigates,  and 
communicates  in  flight,  in  this  order  (Causse  et  ah,  2011;  Taylor,  Bauer,  Poland,  & 
Windell,  2010). 

Field  of  view  (FOV),  It  is  the  spatial  angle  within  which  items  are  visible,  that  is 
degrees  of  visual  angle  relative  to  the  size  of  visual  stimuli.  Specific  to  the  FIUD,  this  the 
angle  within  which  the  symbology  may  be  viewed,  it  is  measured  laterally  and  vertically 
(Newman,  1995).  For  the  purpose  of  this  study,  it  is  a  binocular  overlapping  FOV  and  is 
where  the  two  solid  angles  subtended  at  each  eye  by  the  clear  apertures  of  the  HUD 
optics  intersection  from  a  fixed  head  position  within  the  HEMB  (Harbour,  Hudson,  and 


Zehner,  2011). 
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Focus,  It  is  the  consistency  (lack  of  variance)  in  reaction  time  speed  to  a  change 
in  the  static  and  dynamic  environment  and  reflects  the  sustaining  and  maintaining  of 
attention  (Brain  Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine,  &  Goldman, 
1995;  Sandford  &  Turner,  1994). 

Head  down  display  (HDD),  It  is  an  electronic  device  that  displays  information 
on  a  screen  to  the  pilot  while  looking  down,  however  the  pilot  cannot  see  the  actual 
outside  environment  through  this  type  of  display  (Bailey,  Shelton,  &  Arthur  III,  2011) 

Head  up  display  (HUD),  It  is  a  cockpit  display  that  provides  primary  flight 
information  (PFI)  to  the  pilot  as  he  or  she  looks  through  the  windscreen  (Bailey  et  ah, 
2011;  Newman,  1995).  The  symbology  is  presented  as  a  virtual  image  focused  at  optical 
infinity. 

Helmet  mounted  display  (HMD),  This  is  a  display  mounted  on  the  pilot’s 
helmet  that  performs  the  same  operation  as  the  HUD  (Bailey,  Shelton,  &  Arthur  III, 

2011;  Newman,  2005). 

HUD  Eye  Motion  Box  (HEMB),  It  is  a  three-dimensional  envelope  within 
which  the  pilot’s  eyes  (measured  at  mid-pupil)  need  to  be  in  order  to  accurately  see  100% 
of  the  flight  symbology  (Harbour,  Hudson,  &  Zehner,  2011). 

Information  Display  (ID).  It  is  a  display  to  position  or  expand  for  view,  with  the 
use  of  an  electronic  unit,  a  visual  representation  of  information  in  the  form  of  a  Head  Up 
Display  (HUD),  a  Helmet  Mounted  Display  (HMD),  a  Head  Worn  Display  (HWD),  or  a 
Head  Down  Display  (HDD)  (Bailey,  Shelton,  &  Arthur  III,  2011;  Merriam- Webster, 
2012;  Newman,  1995).  Consequently,  it  is  comprised  of  what  the  right  eye  sees  plus 
what  the  left  eye  sees  from  a  fixed  head  position  within  the  HEMB. 
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Line-of-Sight  (LOS),  It  is,  as  the  pilot  sits  in  Cockpit  Design  Eye  Point  (CDEP), 
a  line  subtending  from  the  pilot’s  mid-pupil  horizontal  to  the  aircraft’s  waterline  centered 
along  the  aircraft’s  boresight. 

Memory.  It  is  the  storage  of  information,  either  short-term  (working  memory)  or 
long-term  (Matlin,  2008;  Klatzky,  1980). 

Mental  workload  (WL).  It  is  related  to  both  demand  and  cognitive  capacity 
(Wickens,  2008).  The  demand  is  imposed  by  tasks  on  the  human’s  limited  mental 
resources,  whether  considered  as  single  or  multiple  (Wickens,  2008).  However,  more 
specifically  mental  workload  for  this  study  is  an  assessment  of  what  proportion  of  mental 
capacity  is  demanded  by  a  task.  Mental  workload  can  be  measured  as  an  implicit 
measurement  of  situation  awareness  (Gutzwiller  &  Clegg,  2012;  Kokar  &  Endsley,  2012; 
Svensson  &  Wilson,  2002;  Wilson  &  Russell,  2003;  Wilson,  et.  al,  2009). 

Neuroergonomics,  It  is  integrated  research  between  psychology,  cognitive 
neuroscience,  human  factors,  engineering,  computer  science,  ergonomics,  and  medicine 
(Eees  et  al.,  2010).  The  purpose  is  to  enhance  knowledge  of  brain  activity,  function,  and 
human  behavior  as  encountered  at  work  and  in  natural  settings  (Parasuraman, 

Christensen,  &  Grafton,  2012). 

Perception,  It  involves  unaware  inference  utilizing  both  biological  and 
psychological  processes  (Blake  &  Sekuler,  2006;  Carlson,  2004).  Items  and  or  events  in 
the  environment  give  off  clues  to  their  existence,  and  the  sensory  organs  detect  these 
clues.  Vision  is  one  of  the  five  senses  and  the  most  involved  when  interfacing  with  a 
HUD  (Wickens,  2008).  The  raw  beginnings  of  perception  begin  with  visual  sensation, 
first  a  stimulus  occurs  (Blake  &  Sekuler,  2006;  Carlson,  2004).  The  human  typically  first 
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mentally  performs  visual  bottom-up  processing  then  a  fraction  of  a  second  later  visual 
top-down  processing. 

Perceptiveness,  It  consists  of  Prudence,  Consistency,  and  Stamina  and  is 
applicable  in  both  the  static  and  dynamic  environment,  involving  both  static  and  dyamic 
visual  processing  (Brain  Train,  2010;  Brain  Train,  2010;  Corbett  &  Constantine,  2007; 
Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner,  1994). 

Pilot,  This  refers  to  the  operator  of  an  aerospace  vehicle  either  manned  or 
unmanned  (USAF,  2007). 

Piloting,  It  is  the  act  of  controlling  an  aerospace  vehicle.  Piloting  involves 
perception  of  flight-relevant  information,  attention,  cognitive  processing,  memory,  and 
motor  responses  to  effect  flight  control. 

Primary  flight  information  (PFI),  It  is  flight  symbology  that  consist  of 
representing  aircraft:  attitude  (pitch  and  bank),  heading,  altitude,  airspeed,  and  vertical 
velocity  (in  some  cases,  this  also  includes  angle  of  attack,  turn  and  slip,  flight  path 
marker,  and  g-meter  indications)  (USAF,  2007). 

Prudence.  It  is  the  selection  or  choice  of  a  correct  response  or  responses  in  a 
dynamic  and  static  environment  (non-inhibited  and  inhibited).  That  is  being  non- 
impulsive,  the  ability  to  not  automatically  react  yielding  an  incorrect  response  in  the 
changing  environment  (Brain  Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine, 
&  Goldman,  1995;  Sandford  &  Turner,  1994) 

Psychological  Refractory  Period  (PRP),  The  fundamental  idea  of  the  PRP  is 
that  a  second  stimulus  can  be  noticed  while  the  first  stimuli  is  still  being  processed, 
however  later  processes  such  as  response  selection  for  the  second  stimuli  cannot  begin 
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until  that  for  the  first  stimuli  is  completed  (Proctor  &  Vu,  2010).  In  this  study,  this  refers 
to  the  degree  to  which  the  stimuli  needed  to  fly  the  airplane  come  from  two  different 
locations  (e.g.,  FOV  and  LOS),  each  of  which  requires  a  cross  check  by  the  pilot.  If  they 
are  in  different  locations  they  will  be  viewed  by  the  pilot  within  a  short  time  interval  of  a 
few  hundred  milliseconds  or  less,  causing  the  response  to  the  second  stimulus  to  be 
delayed  (Proctor  &  Vu,  2010;  Vidulich  &  Tsang,  2012),  thereby  increasing  task 
dificulty. 

Reaction  time.  This  is  the  time  to  respond  to  an  incoming  stimulus,  measured 
from  the  moment  that  it  is  first  perceptible  to  the  moment  a  response  is  made  is  this  time 
(McGrew,  2009). 

Six  degrees  of  freedom.  It  is  the  attitude  of  the  aircraft  (pitch,  yaw,  and  roll)  and 
the  location  of  the  aircraft  (altitude,  heading,  and  spatial  position  [latitude  and  longitude]) 
(Wickens,  2002). 

Situation  awareness  (SA).  Situation  awareness  is  the  psychological  ability  and 
capacity  to  perceive  information  and  act  on  it  acceptably.  In  real-world  terms  for 
aviation,  it  is  the  cognitive  awareness  level  by  the  pilot  as  it  relates  to  aircraft  flight 
(Tsang  &  Vidulich,  2006).  Aircraft  flight  can  be  described  by  six  degrees  of  freedom; 
therefore  an  aspect  of  situation  awareness  is  the  cognitive  awareness  level  by  the  pilot  of 
those  six  degrees  (Wickens,  2002).  Those  six  degrees  also  move  as  a  function  of  time, 
therefore  situation  awareness  also  includes  time  and  change  awareness.  Situation 
awareness  involves  cognition  and  working  memory,  rather  than  action  and  response 
(Wickens,  2002).  It  feeds  on  the  perception  of  the  elements  in  one’s  world  within  a 
volume  of  space  and  time  (Endsley,  2006). 


32 


Speed,  It  is  defined  as  the  average  reaction  time  to  changes  in  the  environment. 
Speed  measures  discriminatory  mental  processing  speed. 

Speed-working  memory  (SWM),  This  is  the  reaction  time-frame  that  includes 
the  processes  of  memory,  visual  scanning,  and  perception.  The  speed  of  responses  are 
measured  in  milliseconds  to  stimuli  for  cognitive  decisions  to  occur  (McGrew,  2009; 
Taylor  et  ah,  2000). 

Stamina.  It  is  the  lack  of  variability  in  a  subject’s  response  time  speed  in  a 
dynamic  and  static  environment.  This  is  the  ability  to  maintaining  a  sustained  effort,  and 
consequently  maintaining  the  speed  of  mental  processing  (Brain  Train,  2010;  Corbett  & 
Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner,  1994). 

Static  Environment,  This  refers  to  a  non-changing  environment  that  requires 
watchfulness  and  vigilance  but  no  overt  responses  from  the  human  operator.  The  human 
operator  is  in  a  state  of  inhibition  when  working  in  this  environment  (Brain  Train,  2010; 
Corbett  &  Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner, 
1994). 

Static  Vision,  It  is  visual  processing  that  occurs  in  the  static  environment  (Brain 
Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995;  Sandford 
&  Turner,  1994). 

Vigilance,  It  is  the  maintenance  of  attention  required  to  respond  to  a  change  in 
the  environment  (a  state  of  inhibition)  such  as  responding  to  a  target.  In  the  environment 
the  mind  can  wander,  the  subject  must  maintain  his  or  her  attention  in  order  not  to  miss  a 
target  (Brain  Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine,  &  Goldman, 


1995;  Sandford  &  Turner,  1994). 
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Visual  attention  (VA),  It  consists  of  human  visual  perceptiveness  and 
attentiveness  (Brain  Train,  2010;  Brain  Train,  2010;  Corbett  &  Constantine,  2007; 
Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner,  1994). 

Visual  or  Visio-  spatial  working  memory  (VSWM),  It  is  WM  that  contains 
visual  and  spatial  information  that  is  stored  in  the  visuospatial  sketchpad  in  the  mind 
(Baddeley,  2006),  this  immediate  memory  can  be  thought  of  as  a  workbench  where 
material  is  continuously  being  combined  and  transformed. 

Working  memory  (WM),  Working  memory  is  a  limited  storage  capacity  of 
memory  of  information  that  cannot  retain  data  for  long  periods  of  time  and  decays  rapidly 
after  one  minute  unless  the  individual  stimulates  other  cognitive  processes  to  retain  the 
information  (MeGrew,  2009). 

Summary 

Static  and  dynamic  visual  processing  is  comprised  of  three  specific  variables: 
visual  attentiveness,  perceptiveness,  and  visuospatial  memory  (Christensen  et  ah,  2013; 
Endsley,  2012;  Vidulieh  &  Tsang,  2012;  Wickens  &  MeCarley,  2008).  Consequently, 
this  project  tested  the  predictive  value  of  these  variables  as  factors  between  a  particular 
task  to  be  performed  and  the  eventual  outcome  of  situation  awareness.  A  field 
experiment  in  the  paradigm  of  neuroergonomics,  (Parasuraman,  Christensen,  &  Grafton, 
2012)  employing  quasi-experimental  repeated-measures  (within-participants)  design,  was 
used  for  this  study.  Further  inquiry  involving  visual  cues  should  clarify  the  role  of  these 
abilities  as  independent  and  or  mediating  variables  in  order  to  make  theory-driven 
predictions  for  situation  awareness  (Harbour  et  ah,  2012;  Vidulieh  &  Tsang,  2012; 
Wickens  &  MeCarley,  2008).  This  affects  society  worldwide. 
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Chapter  2:  Literature  Review 

Introduction 

This  literature  review  is  intended  to  expand  further  on  the  elements  of  existing 
psyehological  theory  and  researeh  that  are  relevant  to  the  work.  Following  an 
introduetion  to  neuroergonomies,  situation  awareness  will  be  reviewed  as  a  central 
concept.  The  discussion  then  proceeds  with  the  basic  component  processes  that  are 
essential  to  situation  awareness,  including  general  cognition,  visual  perceptions  and 
attention,  visuospatial  working  memory,  and  mental  workload.  The  review  will  seek  to 
highlight  where  this  work  was  needed  and  where  it  will  expand  the  body  of  knowledge  in 
these  areas,  and  identify  potential  real-world  implications. 

Documentation 

For  this  40  page  literature  review,  electronic  searches,  in-person  trips  to  university 
libraries,  and  face-to-face  interviews  with  subject  matter  experts  were  completed  and  peer 
reviewed  articles  were  located  using  EBSCOHost,  PsychlNFO,  ProQuest  Research 
Library,  Science  Direct,  Google  Scholar  databases,  the  USAF  AFRL  library,  the  DoD 
DTIC,  the  AFIT,  University  of  Dayton,  and  Wright  State  University  Libraries.  These 
search  engines,  libraries,  and  techniques  were  chosen  for  their  extensive  compendium  of 
appropriate  material  and  information.  Search  terms  included:  workload,  neurology, 
stimuli,  neural  network,  attention,  cognition,  visual  perception,  vision,  human  brain, 
memory,  situation  awareness,  TMSA,  AAT,  MRT,  SEEV,  aviation,  pilot,  cockpit,  UAV, 
display,  HUD,  HDD,  USAF,  aircraft  crashes,  CFIT,  the  human  eye,  brain,  working 
memory,  theoretical  physics,  and  psychological  theory.  Additional  searches  were 
conducted  using  the  reference  sections  of  articles  located  through  electronic  searches. 
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The  literature  review  that  follows  has  been  organized  into  22  sections;  (a) 
Neuroergonomics,  (b)  Cognitive  Situation  Awareness,  (c)  The  Theoretical  Model  of  SA 
(TMSA),  (d)  Critiques  of  the  TMSA,  (e)  Studies  Involving  Vision,  Prediction,  and 
Situation  Awareness,  (f)  Cognition,  Perception,  and  Memory,  (g)  The  HUD,  Situation 
Awareness,  and  Visual  Attention,  (h)  Recent  HUD  Research,  (i)  Work  Load  (WL),  (j) 
Current  Independent  View  of  the  Multiple  Resource  Theory  (MRT),  (k)  Recent  Research 
Measuring  Pilot  Mental  Workload,  (1)  The  Salience  Effort  Expectancy  Value  (SEEV) 
Model,  (m)  The  SA  and  WL  Relationship,  (n)  Vision,  (o)  Attention  and  Motion 
Perception,  (p)  Objective  of  the  Display,  (q)  Relating  Visual  Perception, 
Neuropsychology,  and  SA  (r)  Display  Design,  (s)  Cognitive  Modeling,  and  (t)  Summary. 
Neuroergonomics 

Neuroergonomics  is  a  rapidly  expanding,  interdisciplinary  area  of  research  whose 
purpose  is  to  enhance  knowledge  of  brain  activity,  function,  and  human  behavior  as 
encountered  at  work  and  in  natural  settings  (Parasuraman,  Christensen,  &  Grafton,  2012). 
It  is  integrated  research  between  psychology,  cognitive  neuroscience,  human  factors, 
engineering,  computer  science,  ergonomics,  and  medicine  (Lees,  Cosman,  Lee,  Ericke,  & 
Rizzo,  2010).  Eor  example,  neuroergonomics  can  focus  on  the  psychology  of  perceptual 
and  cognitive  functions  and  actions  in  relation  to  actual  technologies  (Parasuraman  et  ah, 
2012).  A  premise  of  this  multidisciplinary  approach  is  that  knowledge  in  the 
psychological  sciences  will  be  enhanced.  Neuroergonomics:  The  Brain  at  Work 
(Parasuraman  and  Rizzo,  2008)  indicated  this  composite  exploration  is  growing  and  is 
being  driven  by  the  emergence  of  information-saturated  information  display  (ID) 
technology  that  is  now  being  utilized  by  humans  for  activities  requiring  divided  attention 
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and  multitasking  (Parasuraman,  Christensen,  &  Grafton,  2012).  Neuroergonomics 
provides  a  novel  approach  as  a  contemporary  perspective  in  science,  and  field  research  of 
human  situation  awareness  is  an  integral  part  (Lees  et  al.,  2010;  Parasuraman  et  al.,  2012; 
Parasuraman  &  Rizzo,  2008;  Parasuraman,  Sheridan,  &  Wickens,  2002;  Parasuraman  & 
Wilson,  2008;  Wilson,  Estepp,  &  Davis,  2009).  This  work  adopted  a  neuro ergonomic 
approach.  It  is  fueled  by  the  demands  of  data-dense  display  technology,  such  as  the 
information  displays  in  aircraft  cockpits  (HDD,  HUD)  (Parasuraman,  Christensen,  & 
Grafton,  2012). 

Cognitive  Situation  Awareness 

Cognitive  situation  awareness  is  often  discussed  using  philosophical  concepts 
(Blandford  &  Wong,  2004;  Dekker  &  Hollnagel,  2004;  Elliott  et  al.,  2009;  Gorman, 
Cooke,  &  Winner  2006;  Jodlowski,  2008;  Nullmeyer,  Herz,  &  Montijo,  2009;  Rousseau, 
Tremblay,  &  Breton,  2004;  Shelton,  et.  al,  2009).  Eor  example,  Dekker  and  Hollnagel 
(2004)  refer  to  most  situation  awareness  theories  as  nothing  more  than  folk  models.  They 
state  that  contemporary  theories  of  situation  awareness  are  not  sufficiently  articulated  to 
explain  the  details.  Various  theories  superficially  seem  to  be  useful  scientific  models,  yet 
just  below  the  surface  they  lack  an  articulated  mechanism. 

Situation  awareness  is  currently  viewed  as  a  complicated  tangible  that  exists  and 
necessitates  a  keen  sense  for  visual  cues  (Billingsley,  Kuchar,  &  Jacobson,  2001; 

Endsley,  2012;  Parasuraman,  Sheridan,  &  Wickens,  2008).  Endsley’s  (2012)  Theoretical 
Model  of  situation  awareness  (TMSA)  submits  there  are  a  total  of  three  levels  for 
situation  awareness:  a)  perception  (Eevel  1),  b)  comprehension  (Eevel  2),  and  c) 
projection  (Eevel  3).  It  is  difficult  to  define  situation  awareness  in  more  detail  without 
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reference  to  a  particular  task.  For  piloting,  situation  awareness  could  be  described  as  the 
cognitive  awareness  level  by  the  pilot  as  it  relates  to  aircraft  flight  (Tsang  &  Vidulich, 
2003,  2006;  Vidulich  &  Tsang,  2012).  At  the  fundamental  level  of  situation  awareness, 
the  pilot  needs  to  perceive  relevant  information  (Endsley,  1995  a,  1995b)  accurately. 
Pertinent  to  this  study,  situation  awareness  can  be  described  as  a  function  or  product  of 
ID  effectiveness  and  neurocognitive  elements  such  as  mental  work  load  (J.C. 

Christensen,  personal  communication,  Jan  10,  2012). 

The  concept  of  mental  workload  in  the  context  of  this  study  is  the  level  of  demand 
imposed  by  tasks  on  the  pilot’s  limited  mental  resources  (Wickens  &  Hollands,  2000). 
Abstractly,  mental  workload  can  reflect  a  subjective  experience  of  mental  effort  (Dekker 
&  Hollnagel,  2004).  It  is  related  to  both  demand  and  cognitive  capacity  (Wickens,  2008). 
The  demand  is  imposed  by  mental  tasks  on  the  human,  whether  considered  as  single  or 
multiple  (Wickens,  2008).  Workload  in  aviation  is  environment  driven;  therefore,  it  is 
driven  by  the  stimulus  and  activity  involved,  and  it  is  not  something  that  is  necessarily 
self-paced.  In  straightforward  and  simple  terms,  workload  is  the  level  of  mental  effort  a 
given  individual  puts  forth  when  performing  tasks  such  as  aviating,  which  consume 
cognitive  capacity.  In  summary,  workload  is  a  function  of  both  task  difficulty  and 
individual  capability;  as  individual  capability  varies  so  the  same  task  will  produce 
different  levels  of  mental  workload  in  different  individuals  (J.C.  Christensen,  personal 
communication,  Jan  10,  2012). 

Neuroergonomic  methods  provided  the  framework  for  this  field  study  integrating 
subjective  &  objective  measures.  In  keeping  with  the  fundamental  concept  of 
neuroergonomics,  this  study  took  a  multidisciplinary  approach  to  studying  the  brain  at 


38 


work  in  the  cockpit,  including  measuring  workload  as  a  means  to  assess  situation 
awareness  predictors.  Adopting  neuro ergonomic  methods,  this  study  examined  the 
effects  of  ID  design  in  actual  flight  with  neurocognitive  factors  as  intervening  variables. 
In  general,  a  combined  approach  including  psychophysiological  measures  has  been 
identified  as  having  strong  potential  for  improving  our  understanding  of  situation 
awareness  and  workload,  and  the  relationship  between  the  two  (Bailey  et  ah,  2011; 
Campbell,  2010;  Campbell,  Castaneda,  &  Pulos,  2010;  Crawford  &  Neal,  2006;  Douglas, 
Aleva,  &  Havig,  2007;  Flight  Safety  Foundation,  2009;  Geiselman  &  Havig,  2011; 
Harbour,  Christensen,  Estepp,  &  Gray,  2012;  Harbour,  Hudson,  &  Zehner,  2012;  Kim, 
2009;  Kramer  et  ah,  2005;  Wickens,  2008;  Wickens,  Levinthal,  &  Rice,  2010). 

The  Theoretical  Model  of  SA  (TMSA) 

The  Theoretical  Model  of  Situation  Awareness  (TMSA)  as  described  by  Endsley 
(1995  a)  combines  narrative  accounts  of  processes  along  with  the  emphasis  on  empirical 
tests  of  feasibility  (Eigure  1).  Endsley’s  framework  is  grounded  in  hierarchical  levels  of 
situation  awareness  and  is  based  on  information  processing.  Eevel  1  situation  awareness 
contains  the  perception  and  processing  of  cues.  An  example  could  perceive  the 
environment  and  a  display  yielding  spatial  awareness  (Endsley,  2012;  Wickens,  2008; 
Wickens  et  ah,  2008).  Eevel  2  situation  awareness  is  the  comprehension  of  the  current 
situation  by  utilizing  the  information  gained  from  Eevel  1  situation  awareness 
perceptions  combined  with  individual  background  knowledge,  thereby  creating  a 
situation  model  (Endsley,  2000a,  1995a;  Wickens,  2008).  Eevel  3  situation  awareness  is 
the  utilization  of  the  situation  model  to  project  and  predict  the  future  state  (Endsley, 
1995a,  2000b;  Sulistyawati,  Wickens,  &  Chui,  2011;  Wickens,  2009).  Endsley’s  (1995a, 
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2002a)  view  of  situation  awareness  is  often  utilized  as  the  basis  for  research  in  areas  such 
as  system  displays  and  military  operations  (Bid,  Johnsen,  &  Brun,  2004;  Sulistyawati, 
Wickens,  &  Chui,  2011). 

Situation  awareness  may  be  defined  with  greater  specificity  for  a  particular  task  or 
domain;  as  this  study  focused  on  flight  operations,  situation  awareness  was  considered  in 
this  context.  Level  1  situation  awareness  is  the  foundation  for  achieving  situation 
awareness.  In  aviation,  it  begins  with  how  the  pilot  perceives  the  critical  cues  in  the 
environment  (Endsley,  1995a,  1995b,  2012)  needed  to  successfully  fly  and  achieve 
mission  goals.  Aircraft  flight  is  described  by  six  degrees  of  freedom;  therefore,  an  aspect 
of  situation  awareness  is  the  cognitive  awareness  by  the  pilot  of  those  six  degrees 
(Wickens,  2002).  The  six  degrees  of  freedom  in  flight  are  the  attitude  of  the  aircraft  in 
pitch,  yaw,  and  roll,  and  the  location  of  the  aircraft  in  altitude,  latitude,  and  longitude 
(Wickens,  2002).  More  specifically,  because  those  six  degrees  are  always  in  a  constant 
state  of  change  and  projection  into  the  future  is  a  key  part  of  situation  awareness,  then 
full  situation  awareness  would  be  possessing  knowledge  of  the  partial  derivative  of  the 
six  degrees  with  respect  to  time.  This  aircraft  and  aviation  situation  awareness 
knowledge  also  includes  the  awareness  level  of  the  configuration  of  the  aircraft,  and  the 
location  of  other  aircraft  (past,  present,  and  future).  Knowledge  of  the  six  degrees  of 
aircraft  flight  is  considered  primary  flight  information  (PFI)  and  is  conveyed  to  the  pilot 
via  cockpit  information  displays. 
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Figure  1.  Theoretical  Model  of  situation  awareness  (adapted  from  Endsley,  1995b).  The 
three  levels  of  situation  awareness  are  depicted  with  respect  to  individual  capabilities, 
system  capabilities,  and  performance.  This  study  addresses  the  theoretical  links  between 
abilities,  information  processing,  situation  awareness  and  performance. 

Jones  and  Endsley  (1996)  found  that  the  vast  majority  (77%)  of  human  errors  in 
aviation  involving  problems  with  situation  awareness  stem  from  difficulties  with  the 
perception  of  needed  information,  which  is  the  formation  of  Level  1  situation  awareness 
(Eigure  2).  In  the  flying  environment,  pilots  must  make  time  critical  decisions,  therefore 
efficient  information  processing  becomes  paramount  and  the  ID  should  be  designed  so 
that  the  pilot  can  easily  perceive  the  PEI  in  order  to  safely  aviate. 


41 


L«v«l2:  Mtntal  Models 
20% 

Level  1:  Perception  of 
needed  information 

Figure  2.  Errors  in  aviation  attributable  to  situation  awareness  (Jones  &  Endsley,  1996). 
A  signifieant  majority  of  errors  are  assoeiated  with  perception,  which  for  aviation  is 
primarily  visual. 

A  reduction  in  attention  due  to  distractions  or  increased  effort,  which  can  result  in 
increased  workload,  has  been  found  to  undercut  situation  awareness  especially  in  flying 
tasks  and  poses  one  of  the  most  significant  challenges  to  maintaining  situation  awareness 
(Endsley,  2012). 

Critiques  of  the  TMSA 

There  are  at  least  34  different  and  or  varying  definitions  of  situation  awareness 
with  little  consensus  in  the  scientific  community  (Beringer  &  Hancock,  1989;  Breton  & 
Rousseau,  2003;  Cass,  2011;  Jodlowski,  2008;  Rousseau,  Termlay,  &  Breton,  2004)  and 
the  same  existed  for  the  exact  composition  of  situation  awareness  to  include  its  predictors 
(Blandford  &  Wong  2004;  Dekker  &  Hollnagel,  2004;  Elliott  et  ah,  2009;  Gorman, 
Cooke,  &  Winner  2006;  Jodlowski,  2008;  Nullmeyer,  Herz,  &  Montijo,  2009;  Rousseau, 
Tremblay,  &  Breton,  2004).  Therefore,  there  was  not  a  specific  mathematical  equation  or 
variable  relationship  that  included  situation  awareness  predictors.  There  are  at  least  two 
theoretical  alternatives  to  TMSA  (Stanton,  Chambers,  &  Piggott,  2001):  the  theory  of 
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activity  model  to  describe  situation  awareness  (Bedny  &  Meister,  1999)-  and  the 
perceptual  cycle  theory  model  that  is  an  ecological  approach  (Niesser,  1976;  Smith  & 
Hancock,  1995).  These  theories  diverge  in  their  foundational  psychological  construction 
(Salmon  et  ah,  2008).  The  theory  of  activity  model  varies  from  the  TMSA  in  that  it 
describes  situation  awareness  using  eight  functional  blocks  and  posits  that  the  degree  to 
which  blocks  are  engaged  is  contingent  upon  the  character  of  the  task  and  the  individual’s 
goals  (Stanton  et  ah,  2001).  The  perceptual  cycle  theory  model  is  an  alternative  view  of 
situation  awareness,  in  that  it  is  not  dependent  upon  the  domain  or  the  individual,  rather  it 
is  a  function  amidst  the  interaction  of  the  domain  with  the  individual  (Stanton  et  ah, 
2001). 

As  any  important  concept  should,  situation  awareness  has  spawned  some  degree 
of  rigorous  academic  deliberation  (Dekker  &  Hollnagel,  2004;  Dekker  &  Woods,  2002; 
Parasuraman,  Sheridan,  &  Wickens,  2008;  Patrick  &  Morgan,  2010;  Wickens,  2008). 

One  key  point  of  contention  is  whether  situation  awareness  refers  to  the  process  of 
gaining  awareness,  the  product  of  it,  or  a  combination  of  the  two  (Salmon  et  ah,  2008). 
The  answer  to  this  question  then  influences  opinions  on  how  to  best  measure  situation 
awareness;  there  are  a  plethora  of  approaches  to  assessing  situation  awareness  and 
copious  associated  theoretical  debate.  The  continuing  debates  over  situation  awareness 
illustrate  the  need  to  refine  the  theory  (Dekker  &  Hollnagel,  2004;  Parasuraman, 

Sheridan,  &  Wickens,  2008;  Patrick  &  Morgan,  2010;  Wickens,  2008).  Consequently, 
there  simply  was  no  established  theory  as  law  of  situation  awareness;  it  has  been 
described  as  a  theory  as  narrative,  and  therefore,  no  specific  mathematical  equation  or 
variable  relationships  include  situation  awareness  predictors. 
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What  is  known  is  that  situation  awareness  plays  a  vital  role  in  dynamie  deeision- 
making  environments  (St.  John  &  Smallman,  2008).  However,  further  eontroversy 
existed  regarding  specific  details  of  TMSA.  Endsley’s  theory  emphasizes  perception  and 
comprehension  of  the  environment  amid  projection  into  the  future;  however,  it  does  not 
contain  enough  granularity  or  accuracy  in  the  area  of  human  perception  (Elliott  et.  ah, 
2009;  Gorman  et  ah,  2006;  Jodlowski,  2008;  Stanton  et  ah,  2001;  Wickens,  2008).  The 
TMSA  does  not  emphasize  reflective  relationships  between  mental  models  and 
knowledge  of  the  present  system.  There  were  no  specific  neurocognitive  antecedents  or 
predictors  of  SA.  The  theory  is  consequently  incomplete,  lacking,  and  requires  this  for  a 
needed  advancement  (Ellis  &  Eevy,  2009;  Elliott  et  al.,  2009;  Endsley,  2012;  Gillan  et 
al,  2009;  Gugerty,  in  press;  Gutzwiller  &  Clegg,  2012;  Harbour  et  ah,  2012;  Jen-li, 
Ruey-Yun,  &  Ching-Jung,  2013;  Jones,  Connors,  &  Endsley,  2011;  Eau,  Jamieson,  & 
Skranning,  2013;  Proctor  &  Vu,  2010;  Sulistyawati,  Wickens,  &  Chui,  2011;  Vidulich  & 
Tsang,  2012;  Wickens,  2008;  Wickens  &  McCarley,  2008). 

In  order  to  design  better  control  systems  and  information  displays,  it  is  necessary 
to  understand  the  nature  of  situation  awareness  (Blandford  &  Wong,  2004;  Pritchett  & 
Hansman,  2000).  Situation  awareness  formation  should  be  articulated  by  constituent 
psychological  mechanisms  (Dekker  &  Hollnagel,  2004).  Situation  awareness  needs  a 
level  of  detail  to  account  for  psychological  mechanisms  required  to  connect  features  of 
the  sequence  of  events  from  the  start  of  perception  to  the  outcome  of  action  (Dekker  & 
Hollnagel,  2004;  Endsley,  1998,  2006b).  Situation  awareness  is  a  predictor  of  flight 
performance  (Durso,  Truitt,  Hackworth,  Crutchfield,  &  Manning,  1998)  and  enhancing 
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pilot  situation  awareness  improves  flight  performanee  and  allows  for  inereased  safety  and 
efficiency  of  air  and  taxi  operations  (Crawford  &  Neal,  2006). 

In  conclusion,  prior  research  was  incomplete  with  regards  to  the  exact  role  of 
visual  processing  abilities;  current  and  future  research  to  refine  measurement  of  these 
abilities  must  be  accomplished.  Seminal  work  performed  by  Crawford  and  Neal  (2006), 
and  Gorman,  Cooke,  and  Winner  (2006)  indicated  the  need  for  additional  research 
examining  the  impact  of  HUD  usage  on  situation  awareness  in  flight,  in  particular,  with 
respect  to  the  prediction  of  situation  awareness.  The  critical  variables  underlying  the 
formation  of  situation  awareness  as  well  as  the  possible  relationships  among  these 
variables  needed  to  be  identified. 

Studies  Involving  Vision,  Prediction,  and  Situation  Awareness 

Situation  awareness  is  among  the  most  important  cognitive  abilities  required 
successfully  to  perform  complex  cognitive  tasks  such  as  piloting  an  aircraft  or  driving  a 
car  (Tirre  &  Gugerty,  1999).  Case  studies  and  analysis  (Kim,  2009)  of  existing  databases 
have  confirmed  that  a  reduction  in  situation  awareness  is  a  significant  precursor  to 
aviation  performance  failures.  This  section  covers  15  quantitative  experiments  and  ten 
research  articles  that  have  illustrated  the  need  and  attempted  to  discover  the  detailed  links 
between  vision,  cognition,  predictability,  and  situation  awareness  over  a  15-year  period 
up  to  the  present  day,  illustrating  the  continuing  need  to  fill  this  gap  in  the  TMSA.  Tirre 
and  Gugerty  (1999,  2000)  performed  situation  awareness  experiments  that  are  especially 
relevant  to  the  current  work.  In  their  study,  a  driving  simulator  was  utilized  in  an  attempt 
to  derive  a  set  of  individual  cognitive  attributes  that  correlated  with  situation  awareness. 
Participants  completed  a  number  of  scenarios  in  the  highway  driving  simulation  (PC- 
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DriveSim)  that  varied  the  type  of  traffic  hazard  encountered.  The  outcome  yielded  six 
attributes  that  were  correlated  with  a  situation  awareness  composite  score,  together 
accounting  for  fifty-percent  of  the  variance.  The  situation  awareness  composite  score 
was  a  combination  of  accuracy  in  answering  memory  probes  at  and  a  performance  score 
(derived  from  driver  control  responses  to  hazards),  weighted  at  2/3  and  1/3  respectively. 
As  a  result  of  this  weighting,  the  majority  of  the  score  was  based  on  probes  (surveys)  that 
required  the  stopping  of  the  driving  simulator  followed  by  the  participant  answering  a 
series  of  questions  on  two  different  surveys  (based  on  Endsley’s  SAGAT  technique, 
1995b).  The  two  debatably  subjective  survey  measures  were  time-to-passage  (answering 
“which  car”  will  overtake  “which  car”)  and  scene  interpretation  (at  the  end  of  a  moving 
scene  answering  questions  about  other  vehicles’  location  and  who  was  the  erratic  driver). 
The  performance  probe,  which  was  based  on  the  “correctness”  of  actual  driver’s  inputs, 
was  the  only  orthodox  objective  measure. 

In  their  first  experiment,  there  were  88  participants,  with  61  males  and  27  females 
ranging  in  age  from  18  to  30  years.  The  computer-administered  cognitive  battery  CAM 
4.1  measured  the  level  of  working  memory  utilizing  a  digit-span  procedure.  Visual 
processing  ability  was  estimated  by  approximating  search  ability  based  on  three  subtests 
of  the  Air  Force  Officer  Qualification  Test  (AFOQT):  block  counting,  table  reading,  and 
scale  reading.  The  AFOQT  is  a  pencil  and  paper  timed  test  that  estimates  visual  search 
ability,  thus  capturing  a  portion  of  dynamic  visual  processing.  Significant  correlations 
were  found  between  the  situation  awareness  measures  and  working  memory,  visual 
search,  and  temporal  processing.  Specifically,  the  multivariate  analysis  yielded  results  for 
working  memory  as  part  of  a  composite  score  for  situation  awareness  at  r  =  .44,  p  <  .001, 
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correlated  with  both  hazard  deteetion  and  blocking-vehiele  deteetion.  Additionally, 
correlations  existed  between  visual  search  as  part  of  a  composite  seore  for  situation 
awareness  and  hazard  detection,  r  =  .22,  p  <  .05,  and  r  =  .21,  p<  .05  for  blocking-vehicle 
detection.  However,  due  to  a  lack  of  sensitivity  in  the  largely  pencil  and  paper 
measurement  technique  and  limited  set  of  visual  ability  measures;  this  does  not 
eompletely  clarify  the  relationship  between  visual  eapacity  and  situation  awareness. 

Based  on  multiple  regression  analyses,  dynamic  visual  processing,  visual  seareh, 
time-sharing,  and  temporal  processing  were  the  attributes  that  made  signilieant  unique 
contributions.  As  stated  by  the  authors,  however,  preeise  knowledge  of  how,  in  what  way, 
and  or  why  they  affected  situation  awareness  was  and  is  unknown  (Tirre  &  Gugerty, 
1999).  In  this  seminal  work,  the  other  two  discovered  attributes,  static  visual  processing 
and  working  memory,  were  corollary  but  with  “  mixed  evidence,”  (Tirre  &  Gugerty, 
1999,  p.  18)  likely  due  to  chosen  experimental  proeedures  in  the  final  experiment.  The 
preeision  of  the  Tirre  and  Gugerty  study  was  likely  limited  due  to  the  reliance  on  peneil 
and  paper  tests  (e.g.  the  AFOQT  for  visual  abilities  and  ASVAB  for  working  memory, 
circa  1990)  that  could  be  eonsidered  obsolete  for  measuring  those  faetors.  Computerized 
testing  with  precise  reaetion  time  measurement  was  used  in  this  dissertation  in  order  to 
improve  aeeuraey  and  sensitivity  as  compared  to  previous  work. 

In  the  final  experiment,  Tirre  and  Gugerty  added  additional  working  memory 
(quantitative  and  verbal  via  the  pencil  and  paper  ASVAB)  and  visual  processing  tests. 
These  visual  proeessing  tests  included  dynamie  visual  tests  via  a  road  sign  test  and  a 
directions  detection  test  along  with  a  computerized  portion  of  the  AFOQT,  and  static 
visual  testing  via  an  unchanging  stimuli  such  as  searching  for  a  target  in  a  visual  array 
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test,  and  the  pencil  and  paper  AFOQT.  However,  it  fell  short  on  testing  visual  spatial 
working  memory  ability  without  it  being  mixed  with  perceptual-motor  coordination  and 
auditory  testing  via  the  computer-administered  cognitive  battery  CAM  4.1.  Likewise,  the 
final  experiment  did  not  test  basic  visual  processing  elements  such  as  vigilance, 
attentional  focus,  response  inhibition,  and  reaction  time.  For  the  last  experiment,  128 
participants  were  included  with  64  males  and  65  females  ranging  in  age  from  17  to  35 
years  of  age.  Situation  awareness  was  measured  by  the  level  of  driving  performance  in 
the  simulator.  The  working  memory  (WM)  and  situation  awareness  results  as  part  of  the 
multivariate  analysis  were  inconclusive  with  little  to  no  correlation  (r  =  .02,  p>.05),  likely 
because  the  researchers  did  not  measure  WM  in  the  same  manner  as  the  previous 
experiment;  instead,  the  general  aptitude  component  of  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB)  test  was  used.  The  general  aptitude  component  of  ASVAB 
was  used  as  a  proxy  for  WM,  however  the  lack  of  consistent  results  as  compared  with  the 
researchers’  own  previous  experiments  using  visual  WM  tests  suggests  the  general 
aptitude  component  is  not  sufficiently  specific.  In  this  dissertation,  the  positive  results 
obtained  were  likely  contingent  on  the  use  of  precise  cognitive  ability  testing  focused  on 
those  abilities  tapped  by  the  task. 

The  multivariate  correlations  between  situation  awareness  on  one  hand  and 
dynamic  visual  processing  and  static  visual  processing  on  the  other  were  found  to  exist  in 
the  last  experiment,  however  the  correlations  were  not  significantly  different  from  each 
other,  r  =  .29  and  r  =  .18,  respectively,  t(108)  =  .88  p  <  .4.  Tirre  and  Gugerty  (1999) 
indicated  that  even  though  this  research  opened  the  door  to  assessing  candidate  cognitive 
factors  for  situation  awareness,  their  work  was  a  first  exploration,  and  they  hoped  that 
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improved  understanding  of  the  role  played  by  these  abilities  in  forming  and  maintaining 
situation  awareness  would  be  aehieved  in  future  researeh. 

In  order  to  attempt  to  understand  vision's  role  in  SA  formation  further,  Wickens 
(2008),  researehed  particular  characteristics  and  cues  in  a  display  that  can  influence 
situation  awareness.  Wickens  (2008)  investigated  the  effects  of  a  synthetic  vision  system 
(SVS)  and  a  highway- in-the-sky  (HITS)  presented  in  the  pilot’s  cockpit  on  a  display  to 
determine  if  these  cause  change  blindness  in  the  environment  beyond  the  cockpit,  which 
can  be  an  aspect  of  what  is  often  called  attentional  tunneling.  Seven  quantitative 
experiments  and  studies  were  reviewed  by  Wickens  that  utilized  high  fidelity  simulators 
with  instrument  rated  pilots  flying  with  an  SVS  display  with  and  without  HITS. 

Numerous  standard  approaches  and  landings  were  performed,  50%  with  HITS.  The 
percentage  of  pilots  who  failed  to  detect  an  unexpected  outside  environment  event  when 
flying  with  a  SVS  HDD  without  HITS  was  only  17%,  while  with  HITS  it  was  over  twice 
as  large  at  38%.  The  HITS  added  a  very  compelling  3D  command  to  the  display  that 
clearly  was  responsible  for  attention  tunneling  by  the  pilot,  more  so  than  the  SVS  image 
alone,  a  3D  status  display  without  HITS  (Wickens,  2008).  Apparently,  those  pilots  who 
failed  to  detect  an  unexpected  event  lacked  complete  level  1  situation  awareness; 
consequently,  one  may  conclude  that  display  design  and  specifically  attentional  tunneling 
influence  situation  awareness.  However,  the  degree  to  which  improved  visual  attention 
prevents  attentional  tunneling,  hence  improving  situation  awareness,  was  not  addressed  in 
that  work. 

Applied  Attention  Theory  (AAT)  (Wickens  &  McCarley,  2008)  indicates  that 
visual  attention  control,  scanning,  information  sampling,  visual  search,  spatial  attention 
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and  displays  play  a  role  in  pilot  mental  workload,  whieh  in  turn  would  imply  an  influenee 
on  situation  awareness  as  well.  Wiekens  (2008)  diseovered  that  visual  seareh  and 
attention  were  unique  faetors  for  workload.  An  understanding  of  visual  proeessing 
factors  operating  in  dynamic  environments  related  to  attention  is  incomplete  (Wiekens  & 
McCarley,  2008,  p.  38). 

Elliott  et  al.  (2009)  performed  a  deep  dive  assessment  of  quantitative  research 
bringing  into  play  visual  displays,  information  providing  auditory  modalities,  and 
vibrotactile  interaction.  Even  though  this  dissertation’s  focus  is  on  vision  only  IDs,  the 
Elliot  deep  dive  assessment  brought  to  light  gaps  and  needs  in  the  area  of  vision  and 
situation  awareness.  Studies  meeting  assured  criteria  were  reviewed  and  evaluated,  such 
as  for  research  design  and  characteristics,  and  cue  information  complexity  for  spatial 
awareness.  One  of  the  comparisons  that  mark  this  study  relevant  and  needed  is  the 
comparisons  between  visual  cues  and  a  multimodal  combination  of  tactile  and  visual 
cues,  or  tactile  cues  alone  representing  the  same  information. 

One  of  the  Elliot  et  al.  (2009)  hypotheses  indicated  that  tactile  cues  would  not  be 
effective  when  replacing  visual  cues,  and  this  was  to  some  extent  supported.  Due  at  least 
in  part  to  the  conservative  random  effects  model  used  for  statistical  testing,  the  results 
were  not  significant.  Even  though  the  effect  of  replacing  visual  cues  with  tactile  cues 
was  large  (g  =  0.95,  SE  =  0.22),  significant  variation  prevented  a  statistically  significant 
result  (Qtotal  =  288.89, <  0.01).  This  recent  finding  by  Elliot  reemphasized  that  vision 
is  the  most  dominant  sense  when  it  comes  to  influencing  situation  awareness  (Wade  & 
Swanston,  2012),  consequently  further  elucidating  the  visual  attributes  of  situation 
awareness  will  have  the  greatest  impact. 
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Elliot  et  al.  (2009)  provide  additional  knowledge  to  build  upon  when  researehing 
visual  cuing,  displays,  and  the  neurocognitive  factors  involved  in  Level  1  situation 
awareness  prediction.  This  study  identified  areas  in  need  of  further  investigation, 
specifically  work  on  mediating  factors  and  refining  guiding  principles  to  determine  when, 
where,  why,  and  or  how  cues  come  into  play  to  support  human  performance  in 
demanding  or  complex  environments.  Future  studies  need  to  further  investigate  and 
refine  theory-driven  predictions  for  workload  and  demands  for  attentional  abilities  to 
include  visual  cuing  (Elliot  et  al,  2009;  Strater,  Riley,  Faulkner,  Hyatt,  &  Endsley, 

2006).  As  concluded  by  Wickens  (2008),  additional  experiments  should  be  performed 
examining  individual  human  differences,  principally  with  regard  to  attention  skills,  and 
task  performance  as  related  to  situation  awareness  and  workload. 

The  research  performed  by  Jen-li,  Ruey-Yun,  and  Ching-Jung  (2013)  examined 
display  design  for  unmanned  aerial  vehicle  (ElAV)  monitoring,  and  its  effects  on  operator 
situation  awareness,  performance,  and  mental  workload.  ElAV  monitoring  can  be 
demanding  on  visual  attention.  This  quantitative  research  involved  56  participants  that 
were  randomly  selected  and  assigned  to  either  a  situation-augmented  display  or  a 
conventional  display  condition  to  work  on  UAV  monitoring  tasks.  The  operator  in  UAV 
flights  has  to  rely  primarily  on  vision  in  this  agent-based  system,  consequently  the 
question  remains  to  be  what  role  does  display  design  and  human  visual  abilities  play  in 
the  human-robot  interface  in  order  to  enhance  situation  awareness  (Jones,  Connors,  & 
Endsley,  2011),  again  illustrating  the  need  to  expand  the  TMSA.  The  purpose  of  this 
study  was  to  examine  these  effects  in  the  UAV  monitoring  environment  (Jen-li,  Ruey- 
Yun,  &  Ching-Jung,  2013). 
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College  students  volunteered  to  partieipate  in  the  study;  they  were  not  trained 
aviators,  UAV  operators,  or  USAF  pilots  and  thus  we  may  expeet  significant  variance  in 
prior  experience  and  expertise.  This  may  have  been  a  factor  contributing  to  their  need  for 
a  relatively  large  sample  of  56  individuals.  Fifty  men  and  six  women  provided  an  equal 
number  of  observations  in  each  display  condition.  The  average  age  of  the  participants 
was  21.82  (SD  =  4.03,  range:  16-33).  For  these  experiments,  there  were  two  display 
types  utilized:  the  conventional  display  vs  a  situation  enhanced  and  augmented  display. 
The  conventional  display  served  as  a  comparison  or  control  for  the  situation-augmented 
display.  The  conventional  display  system  was  modeled  after  the  current  FAA  display 
models.  The  situation-augmented  display  incorporated  a  projected  flight  trajectory  to  aid 
visual  processing. 

The  study  sought  to  measure  the  difference  in  operator  performance  as  a  function 
of  the  two  different  displays.  Abnormal  events  were  randomly  presented  to  the 
participant,  and  then  corresponding  errors  were  measured.  UAV  operator  error  rate  was 
submitted  to  a  4-way  ANOVA  with  repeated  measures.  This  analysis  showed  that  only 
the  main  effect  of  number  of  abnormal  events  {F  3452  =  28.46, <  .001,  p  2  =  .35)  and 
block  (F  2,108  =  30.74, />  <  .001,  p  2  =  .36)  were  significant.  The  main  effect  of  block 
could  be  due  to  learning  effects  within  the  study,  a  possible  consequence  of  the  non¬ 
expert  participants.  As  expected,  the  error  rate  increased  from  48.4%  (SE  =  .016),  50% 
(SE  =  .016),  50.5%  (SE  =  .016),  to  55.7%  (SE  =  .016)  as  the  number  of  abnormal  events 
increased  from  0,  1,  2,  to  3,  illustrating  that  the  manipulation  of  workload  (task  difficulty) 
was  successful.  The  only  statistically  significant  effect  related  to  display  type  was  a 
display  type  by  number  of  abnormalities  by  block  interaction  {F  6,324  =  2.93,  p<  .01 1,  p  2 
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=  .05).  In  order  to  test  the  statistieally  significant  three-way  interaction  of  display  type 
by  number  of  abnormalities  by  block  on  the  error  rate,  separate  two-way  ANOVAs  were 
performed  for  each  block.  The  only  statistically  significant  effect  regarding  the  display 
type  error  rate  was  a  display  type  by  the  number  of  abnormalities  relation  {F  2.959, 

160.327  =  6.06,;?  <  .001,  p  2  =  .101). 

Nonetheless,  multiple  comparisons  between  means  showed  no  significant 
differences  between  display  types  in  the  different  number  of  abnormalities  conditions. 
The  hypothesis  that  the  situation-augmented  display  would  decrease  the  error  rate  in  a 
UAV-monitoring  task  was  not  supported.  However,  and  as  a  very  important  note  the 
error  rate  remained  near  50%  after  24  trials  of  practice.  This  high  error  rate  suggests  that 
the  UAV-monitoring  task  was  extremely  hard  for  these  novice  participants  (as  would  be 
flying  an  aircraft).  It  may  have  been  possible  to  observe  more  of  the  potential  benefit  of 
the  situation-augmented  display  if  participants  had  been  more  experienced.  Clearly,  this 
shows  that  using  a  trained  homogeneous  group  such  as  USAF  pilots  would  have 
produced  results  that  were  more  realistic  and  meaningful,  while  requiring  a  smaller 
sample  size.  For  this  study,  flight  completion  time  was  also  evaluated  (the  faster,  the 
better),  as  a  function  of  display  type  used.  Results  of  a  4-way  ANOVA  supported  all  the 
predictions:  the  main  effects  of  display  type  {F  1,54  =  A.%l\,p  <  .032,  p  2  =  .08),  the 
number  of  abnormal  events  (  F3462  =  22.14, />  <  .001,  p  2  =  .29),  the  number  of  light 
signals  (  F  1,54  =  14.06, />  <  .001,  p  2  =  .008),  and  the  block  ( F2,io8  <  40.21, />  <  .001,  p  2 
=  .43)  were  all  statistically  significant.  The  main  effect  of  display  type  supports  the 
hypothesis  that  the  situation-augmented  display  would  improve  UAV-monitoring 
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performance  in  terms  of  shortened  flight  completion  time  {M=  100.37  sec./trial,  SE  = 
3.68),  when  compared  to  the  conventional  display  (M=  1 1 1.85  sec./  trial,  SE  =  3.68). 

Most  notably,  the  main  effect  of  display  type  indicated  that  participants 
using  the  situation-augmented  display  were  2.67  sec.  faster  in  abnormality  detection  (M  = 
5.67  sec.,  SE  =  0.50)  than  those  using  the  conventional  display  (M=  8.36,  SE  =  0.50), 
suggesting  that  the  effects  of  situation-augmented  display  on  abnormality  detection  were 
robust  across  different  workloads  and  noise  levels. 

In  regards  to  mental  workload,  results  of  the  t-tests  on  participants'  un-weighted 
total  scores  and  the  six  sub-scores  on  NASA-TLX  questionnaire  (Charlton  et  ah,  2002) 
showed  no  statistically  significant  difference  between  the  two  types  of  displays  (C  s  < 

1 .55).  In  general,  the  UAV-monitoring  task  appeared  to  place  a  higher  load  on  mental 
demands  (M=  \1 .16,  SD  =  2.37),  effort  demands  (M=  15.36,  SD  =  3.56),  and  speed 
demands  (M=  14.80,  SD  =  3.76)  than  performance  demands  (M=  1 1.09,  SD  =  4.69), 
physical  demands  (M=  1 1.07,  SD  =  5.71),  and  frustration  experienced  (M=  10.02,  SD  = 
6.77).  Reducing  UAV  operators'  mental  workload  is  thus  important  in  the  UAV- 
monitoring  task.  The  situation-augmented  display  was  not  sufficient  for  alleviating  the 
operator's  mental  workload  in  UAV  monitoring. 

These  results  illustrated  that  compared  to  the  conventional  display,  the  effects  of 
situation-augmented  display  on  flight  completion  time  and  abnormality  detection  time 
were  robust  across  different  workloads  but  error  rate  and  perceived  mental  workload 
were  unaffected  by  the  display  type.  With  the  increasing  complexity  of  new  automation 
technology,  discovering  the  correct  visual  processes  to  aid  the  control  operator’s  situation 
awareness  remain  a  significant  challenge  for  the  field.  An  important  point  gained  from 
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this  study  is  that  the  Level  1  SA  from  the  TMSA  still  presented  problems  for  these 
researehers  in  that  they  had  difficulty  in  measuring  the  effects  between  operator  visual 
abilities  and  display  usability  (Jen-li  et  ah,  2013).  More  studies  are  needed  to  address 
these  unresolved  and  outstanding  issues  (Jen-li  et  ah,  2013)  expanding  the  TMSA. 

Objective  measures  of  situation  awareness  and  workload,  conceivably  in  actual 
flight,  would  most  likely  provide  the  basis  for  this  deeper  understanding  of  situation 
awareness  (Harbour  et  al.,  2012),  as  will,  testing  for  participant’s  levels  of  visual  abilities 
beforehand.  Given  these  findings,  the  most  important  predictive  measures  for  future 
studies  would  be  dynamic  visual  attention  and  spatial  tests.  For  the  past  decade, 
researchers  have  been  attempting  to  unravel  these  important  mysteries.  Tirre,  Elliot, 
Wickens,  Parasuraman,  Christensen,  Vidulich,  Tsang,  Jen-li,  Gugerty,  and  many  others 
have  been  performing  research  to  further  solve  this  mystery  and  with  each  successive 
article,  the  field  has  gotten  closer  to  the  answers,  but  there  was  more  to  be  done  (Endsley, 
2012;  Vidulich  &  Tsang,  2012). 

Cognition,  Perception,  and  Memory 

In  a  broad  sense,  cognition  is  thought  or  collective  thoughts;  more  specifically,  it 
is  the  human  process  that  is  mental  activity,  which  utilizes  thoughts  for  acquiring, 
processing,  storing,  transforming,  and  using  knowledge  (Matlin,  2008).  Elements  of 
cognition  include  sensation  and  perception,  imagery,  memory,  reasoning,  and  problem 
solving  (Blake  &  Sekuler,  2006). 

Situation  awareness  does  require  input  processing  such  as  cue  recognition,  in 
order  to  make  a  situation  assessment  and  prediction  upon  which  good  choices  can  be 
based  (Artman,  2000;  Parasuraman,  Russo,  Wickens,  2008),  and  cue  recognition  involves 
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cognition,  perception,  and  memory  processes.  Durso  et  al.  (2007)  point  out  that  through 
the  bottom-up  proeess  of  pereeption,  information  in  the  environment  is  indieated  by  cues, 
followed  by  the  top-down  process  resulting  in  the  development  of  a  situational  model 
integrating  this  into  context.  The  top-down  proeess  involves  a  higher  order  of  cognitive 
thought  utilizing  visual-spatial  working  memory  (Ericsson  &  Delaney,  1999;  Eriesson, 
Patel,  &  Kintsch,  2000;  Hameed,  Jayaraman,  Ballard,  &  Sarter,  2007).  In  order  to 
construct  a  representation  of  the  environment,  humans  must  develop  an  event-base 
around  them  (Durso  et  ah,  2007).  Therefore,  if  situation  awareness  develops,  in  the  same 
way,  as  discourse  comprehension,  event-base  development  begins  through  a  strictly 
bottom-up  process  driven  by  sensory  information.  Therefore,  for  Eevel  1  in  situation 
awareness  (pereeption)  to  occur  correctly  it  is  vital  to  the  initial  stage  of  visual  perception 
and  the  processing  of  that  information  to  also  occur  properly.  Top-down  processing  is 
utilized  to  allow  assigning  the  appropriate  meaning  in  a  particular  context  to  the  visual 
information,  whieh  requires  WM,  specifically  visual-spatial  WM  (Durso  et  ah,  2007; 
Matlin,  2008)  for  spatial  information. 

This  would  entail  three  variables:  visual  perceptiveness,  visual  attentiveness,  and 
visual-spatial  working  memory  (WM)  (Blake  &  Sekuler,  2006;  Brain  Train,  2010; 

Matlin,  2008;  Carlson,  2010).  These  may  be  summarized  as:  a)  did  the  pilot  see  it  at  all, 
b)  did  the  pilot  attend  to  it,  and  c)  did  the  pilot  encode  it  in  memory  sueh  that  he  or  she 
could  use  it  in  building  a  situation  model.  An  understanding  of  visual  processing  factors 
(such  as  independent  or  moderating  or  mediating/intervening  variable/s)  operating  in 
dynamie  environments  was  incomplete  (Crawford  &  Neal,  2006;  Elliott  et  ah,  2009;  Tirre 
&  Gugerty,  1999;  Wiekens,  2002).  This  suggested  that  the  inherent  neuroeognitive 
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factors  of  visual  attention  (perceptiveness  and  attentiveness)  and  visual-spatial  working 
memory,  as  part  of  a  theory  may  explain  their  support  for  the  development  of  situation 
awareness. 

Working  memory  has  a  relatively  small  capacity  and  only  contains  a  small 
amount  of  information  that  the  human  is  actively  using  at  the  time,  in  relation  to  using  it 
to  perform  work  (Zook,  Davalos,  DeLosh,  &  Davis,  2004;  Vieente,  1992).  Working 
memory  can  be  lost  within  thirty  seeonds  unless  it  is  repeated  (Charness,  1976;  Chase  & 
Simon,  1988;  Matlin,  2008).  Baddeley  (2006)  indicates  working  memory  performance 
influenees  the  ability  of  the  eentral  executive  to  manage  work  on  two  tasks  at  the  same 
time,  or  to  inhibit  a  task  response.  Working  memory  eapacity  must  be  shared  between 
current  activities  and  temporary  storage  of  intermediary  results  and  recently  eneoded  data 
(Copeland  &  Radvansky,  2004;  Daneman,  &  Carpenter,  1980).  Its  capacity  varies 
considerably  aeross  individuals,  and  it  is  the  eentral  bottleneck  in  information  processing 
(Engle,  Carullo,  &  Collins,  1991).  Aecording  to  Baddeley,  visual  and  spatial  information 
is  stored  in  a  limited  visuospatial  sketehpad;  performing  two  visuospatial  tasks 
simultaneously  will  eause  interferenee  with  eaeh  other  (Sohn,  &  Doane,  1997,  2000, 
2003).  Therefore,  visuospatial  working  memory  should  be  eritical  to  situation  awareness. 
The  HUD,  Situation  Awareness,  and  Visual  Attention 

As  discussed  above,  Endsley  stated  (1995a)  that  the  pereeption  of  the  elements  in 
the  environment  is  the  first  level  in  situation  awareness.  In  flying  tasks,  the  pilot  has  to 
reference  the  eockpit  Information  Display  (ID)  to  pereeive  and  comprehend  primary 
flight  information.  A  HUD  should  enhance  pilot  pereeption  (level  1)  and  comprehension 
(level  2  situation  awareness)  of  the  environment  to  faeilitate  projection  of  his  or  her  status 


57 


in  the  near  future  (level  3  situation  awareness),  (Crawford  &  Neal,  2006;  Flight  Safety 
International,  2009;  Gorman,  Cooke,  &  Winner,  2006).  The  foundation  for  aehieving 
situation  awareness  in  aviation  thus  begins  with  how  the  pilot  pereeives  the  critical  cues 
in  the  environment  (Endsley,  1995  a,  1995b)  to  fly  successfully  and  achieve  mission 
goals.  To  enhance  situation  awareness,  the  ID  design  should  be  salient  and  easy  to  see  in 
order  to  minimize  demands  on  the  pilot’s  attentional  abilities.  Effective  ID  design 
should,  therefore,  benefit  overall  task  performance,  though  this  effect  is  expected  to  be 
mediated  by  pilot  visual  skills.  Gorman,  Cooke,  &  Winner  (2006)  and  Crawford  &  Neal 
(2006)  indicate  that  future  research  needs  to  uncover  the  factors  that  predict  situation 
awareness  and  discover  how  they  interact  with  pilot  abilities  in  apprehending  the 
information  offered  by  cockpit  IDs. 

A  subjective  evaluation  derived  from  flight  simulator  data  (Elight  Safety 
Eoundation,  2009;  Kim,  2009)  suggests  that  many  of  the  accidents  attributed  to  CEIT 
may  have  been  prevented  had  a  HUD  been  installed  in  the  cockpit.  That  study  assessed 
what  may  have  been  a  likely  outcome  assuming  that  a  properly  operating  HUD,  correctly 
operated  by  a  trained  crew,  had  been  in  use  on  each  accident  aircraft  (Elight  Safety 
Eoundation,  2009).  HUDs  are  intended  to  lower  workload  and  increase  situation 
awareness  (ESE,  2009).  The  proper  operation  of  the  HUD  system  may  be  impaired  due 
to  poor  placement  and  visibility,  particularly  in  aircraft  that  did  not  originally  come  with 
a  HUD  (Hudson,  Zehner,  Harbour,  &  Whitehead,  2011).  This  study  included  conditions 


both  with  and  without  a  HUD  available. 
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Recent  HUD  Research 

The  HUD  manipulation  was  incorporated  into  this  study  as  a  real-world  means  of 
varying  task  difficulty  with  consequent  impacts  on  situation  awareness.  Harbour, 
Hudson,  and  Zehner  (2012)  performed  research  on  head  up  display  (HUD)  placement  to 
determine  pilot  visibility.  Based  on  physical  characteristics  of  the  HUD  and  a 
distribution  of  pilot  eyes  around  the  design  eye  point,  the  loss  of  information  visibility 
was  mapped  and  predictions  made  regarding  possible  psychological  refractory  periods 
induced  by  switching  attention  between  displays  (see  Table  1  Pilot  Visibility  (View)  & 
Table  2  Psychological  Refractory  Period). 

Table  1 


Pilot  Visibility  (View) 


FOV 

LOS 

HUD 

No  Loss 

Up  to  5  degrees  up 

HDD 

No  Loss 

35  degrees  down 

Table  1  illustrates  the  FOV  and  LOS  as  a  function  of  display  used. 


Table  2 


Expected  Psychological  Refractory  Period 


FOV 

LOS 

HUD 

No  PRP 

No  PRP 

HDD 

No  PRP 

PRP 

Table  2  illustrates  the  PRP  associated  with  the  display  used  as  a  function  of  FOV  and 
LOS. 
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The  study  recommended  that  more  research  in  this  area  should  be  accomplished, 
utilizing  neuroergonomic  techniques,  in  the  actual  aircraft  in-flight.  The  effects  of 
varying  spatial  locations  of  information  displays  (ID)  in  addition  to  individual  differences 
in  visual  perception  and  attention  abilities  coupled  with  the  effects  on  pilot  workload  and 
situation  awareness  should  be  researched  (Harbour  et  ah,  2012;  Vidulich  &  Tsang,  2012; 
Wickens  &  McCarley,  2008). 

Work  Load  (WL) 

The  Federal  Aviation  Administration  defines  mental  workload  as  the 
psychological  and  physiological  demands  that  occur  on  a  human  while  performing  a  task 
or  multiple  tasks  (FAA,  2011).  Mental  workload  can  be  conceptualized  as  being  in  one 
of  the  two  regions  of  task  demand  level  (Wickens  &  Hollands,  2000).  The  first  is  when 
the  demand  is  less  than  the  available  capacity  of  resources,  therefore,  there  are  resources 
not  used  in  the  task  performance.  The  second  region  is  one  in  which  the  demand  exceeds 
the  capacity;  therefore,  performance  will  break  down  (Wickens,  2008).  The  challenge 
lies  in  determining  the  boundary  between  these  regions  (Wickens,  2008;  Yerkes  & 
Dodson,  1908). 

Flying  a  modem  military  aircraft  is  indeed  operating  a  complex  system, 
performing  several  tasks  at  the  same  time  that  increase  pilots’  mental  workload. 

Accurate  mental  workload  measurement  is  critical  because  workload  significantly  affects 
human  performance,  and  a  better  understanding  of  the  relationship  between  primary 
abilities,  mental  workload,  and  situation  awareness  in  turn  could  prove  essential  in 
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preventing  human  faetors -related  aviation  mishaps  (Kang,  2008;  Kang,  Yuan,  Liu  ,  & 

Liu,  2008). 

Multiple  Resource  Theory  (MRT)  by  Wickens  (1984,  2002,  2008)  is  a  prominent 
theory  of  mental  workload  and  aids  in  the  prediction  of  success  in  multi-tasking.  MRT  is 
based  on  three  major  factors:  1)  Resource  Demand  (task  difficulty),  2)  Resource 
Structure  (neuropsychological  characteristics  of  the  individual  brain),  and  3)  Dual  Task 
Decrement  (e.g.  which  task  suffers  when  two  are  performed  concurrently).  In  the  area  of 
neurophysiological  /  neurocognitive  characteristics  that  may  underlie  MRT,  more  work 
needs  to  be  done  (Wickens,  2008).  The  research  for  this  dissertation  specifically 
investigated  neurocognitive  Resource  Structure  characteristics  such  as  visual 
attentiveness,  visual  perceptiveness,  and  visual-spatial  working  memory  as  independent 
variables  and  the  effects  on  pilot  workload  and  situation  awareness  (Harbour  et  ah,  2012; 
Kim,  2009;  Endsley,  2012;  Sulistyawati,  Wicken,  Chui  &  201 1;  Vidulich  &  Tsang, 

2012). 

Current  Independent  View  of  the  Multiple  Resource  Theory  (MRT) 

MRT  has  emerged  as  an  important  research  topic  in  the  past  five  years  (Bulkley  et 
al,  2009;  Lei  &  Roetting,  2011;  Pickel  &  Staller,  2012;  Sarter,  2012;  Vidulich  &  Tsang, 
2012)  and  it  is  influencing  the  design  of  future  information  displays  (Sarter,  2012). 
Multiple  resource  theory  is  posited  as  an  alternative  explanatory  model  to  a  single 
resource  theory  (Embrey,  Blackett,  Marsden,  &  Peachey,  2006).  In  the  views  of  Sarter 
(2012)  and  Embrey  et  al.  (2006),  MRT  is  principally  for  modeling  task  difficulty 
(similarity  level  for  multiple  tasks)  and  human  performance  based  on  cognition  utilizing 
groups  of  mental  resources.  MRT  has  shown  to  be  effective  in  this  area;  empirical 
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studies  found  r  =  0.98  and  an  r  =  0.92  in  predicting  hazard  response  time  and  in-vehicle 
task  performance,  respectively  (Sarter,  2012;  Vidulich  &  Tsang,  2012).  At  first,  it  was 
emphasized  that  MRT  was  intended  to  predict  the  outcome  of  one  or  two  time-shared 
continuous  tasks  at  an  elementary  level  (Sarter,  2012).  However,  later  in  the  field, 
neuroergonomic  evidence  has  accumulated  suggesting  that  both  visual  and  auditory 
modes  can  mix  to  either  limit  and  or  enhance  multimodal  information  processing  (Sarter, 
2012;  Vidulich  &  Tsang,  2012). 

MRT  is  evolving  and  feeding  application  models  as  tools  for  improving  interface 
design  (Sarter,  2012).  As  discussed  by  Lei  and  Roetting  (2011),  Wickens  has  made 
advancements  to  MRT  both  in  2002  and  again  in  2008.  These  advancements  have 
expanded  MRT  while  adding  detail  and  granularity.  Current  MRT  submits  that  there  are 
limitations  to  attention  resources  as  before.  Among  other  changes,  current  evidence 
supports  that  two  aspects  of  visual  processing,  referred  to  as  ambient  and  focal  (foveal) 
vision,  appear  to  sometimes  draw  on  separate  resources,  therefore  tasks  that  draw  on  both 
aspects  may  result  in  relatively  improved  performance,  while  simultaneous  tasks  that  load 
on  only  one  aspect  are  likely  to  degrade  performance  when  these  processes  occur 
simultaneously  (Sarter,  2012).  An  important  characteristic  of  this  view  of  MRT  is  the 
proposal  that  the  effect  of  task  demand  and  resources  may  not  be  just  qualitative  but  may 
also  involve  quantitative  relationships  (Embrey  et  ah,  2006;  Lei  &  Roetting,  2011; 
Padgett,  2004;  Sarter,  2012;  Vidulich  &  Tsang,  2012). 

Recent  Research  Measuring  Pilot  Mental  Workload 

A  recent  study  conducted  by  Harbour,  Christensen,  Estepp,  and  Gray  (2012)  was 
primarily  application-based  research  to  ensure  that  cardiac  activity  could  indeed  be  taken 
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in-flight  and  be  reflective  of  pilot  mental  workload  when  looking  at  heart  rate  (HR)  and 
HR  variability  (HRV).  An  exploration  of  the  workload  protocol  and  a  preliminary 
assessment  of  pilot  workload  and  situation  awareness  during  flight  was  accomplished,  as 
a  trial  study  to  improve  the  process  and  verify  that  it  could  be  done  using  in  flight  heart 
rate  recording.  Initial  comparisons  were  made  between  the  Head  Down  Display  (HDD), 
and  the  original  Head  Up  Display  (HUD)  configuration  while  pilots  performed  various 
training  maneuvers  (Harbour  et  ah,  2012). 

This  was  a  quasi-experimental  study,  with  varying  numbers  of  pilots  represented 
in  each  comparison.  Five-minute  segments  during  each  maneuver  were  analyzed  by 
extracting  the  average  HR  and  standard  deviation  of  the  R-R  interval  as  a  HRV  measure 
(Harbour  et  ah,  2012).  It  was  noted  that  the  psychophysiological  measurement  of  HRV, 
which  is  objective,  provided  more  granularity  and  significant  differences  in  workload  as  a 
function  of  task  and  ID  used  than  subjective  surveys  (Harbour  et  ah,  2012).  This  work 
refined  a  neuroergonomic  protocol  for  in-flight  workload  assessments  and  demonstrated 
that  it  is  effective  in  this  environment. 

The  addition  of  the  objective  physiological  measurement  of  heart  rate  augments 
the  veracity  and  robustness  of  the  findings  and  provides  a  more  complete  picture  of  pilot 
workload  and  situation  awareness  (Harbour  et  ah,  2012).  Further  research  in  this  area 
needs  to  be  accomplished,  such  as  adding  electroencephalogram  (EEG)  data  collection, 
and  testing  for  the  effects  of  varying  spatial  locations  of  information  displays  (ID) 


(Harbour  et  ah,  2012). 


63 

This  completes  the  review  of  applicable  research  on  mental  workload;  we  will 
next  turn  to  a  model  of  attention  that  links  Multiple  Resource  Theory  to  display  usability, 
operator  performanee,  and  situation  awareness  (and  the  TMSA). 

The  Salience  Effort  Expectancy  Value  (SEEV)  Model 

A  theoretical  model  of  selective  attention  called  the  Salience  Effort  Expectancy 
Value  (SEEV)  model  links  basic  research  psychological  models,  salience  and  attention 
capture,  engineering  models,  and  expected  value  optimization  (Wickens  &  McCarley, 
2008).  An  example  of  SEEV’s  purpose  could  be  to  improve  ID  design  in  order  to 
counteract  inattention  (Ma  &  Kaber,  2007;  Matthews,  Bryant,  &  Webb,  2001). 

The  SEEV  is  linked  to  MRT  primarily  through  visual  modality  and  it  utilizes 
MRT  in  the  creation  of  an  attention  based  applications  tool.  More  specifically,  SEEV 
uses  foveal  vision  as  attention  directly  from  MRT.  The  parameters  of  the  SEEV 
(Wickens  &  McCarley,  2008),  which  drive  visual  attention  around  the  environment,  are: 
(1)  Salience  (S),  the  exogenous  attention  capturing  properties  of  events,  e.g.  bright 
symbols  that  are  salient  on  the  ID,  (2)  Effort  (Ef)  that  inhibits  the  movement  of  attention 
across  longer  distances,  (3)  Expectancy  (Ex),  the  probability  of  seeing  an  event  at  a 
particular  location,  an  endogenous  cognitive  factor  that  is  calibrated  by  the  individual  to 
the  frequency  of  events  that  occur  at  that  location,  and  (4)  Value  (V),  which  is  the 
importance  (value)  of  tasks  provided  by  the  attended  event,  in  addition  to  the  relevance  of 
the  event  to  a  valued  task.  Therefore,  the  Probability  of  Attending  P(A)  =  s*S  -ef*EP  + 
(ex*EX  +  vV)  or  (ex*EX  *  vV).  Wickens  and  McCarley  (2008)  found  the  SEEV  model 
to  have  a  correlation  R-value  of  0.78  for  P(A)  and  actual  attending,  with  a  model  fit  (R) 
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>0.90.  MRT  is  the  fuel  for  the  SEEV,  and  the  SEEV  is  the  filter  application  (Wickens  & 
McCarthy,  2008). 

With  the  application  of  the  SEEV  model,  aircraft  cockpits  can  be  designed  such 
that  the  location  and  appearance  of  symbology  on  the  display  more  efficiently  correspond 
with  the  actual  environment  in  the  most  efficient  prioritized  manner.  What  the  SEEV 
model  lacked  was  the  influence  of  individual  ability  on  visual  attention  (individual  as 
“Eiveware”;  Edwards,  1972),  i.e.  the  levels  of  the  individual’s  abilities  in  visual 
attentiveness,  visual  perceptiveness,  and  visual  spatial  working  memory.  Eurther 
individualization  of  SEEV  would  advance  the  theory  by  elucidating  the  effects  of 
individual  differences  in  visual  abilities  on  resultant  visual  attention,  which  could 
conceivably  result  in  individually  customizable  information  displays  that  maximize 
usability  improving  SA,  linking  the  SEEV  and  TMSA. 

The  SA  and  WL  Relationship 

Task  difficulty  has  been  demonstrated  to  influence  mental  workload,  and  mental 
workload  in  turn  affects  cognitive  situation  awareness  (Svensson  &  Wilson,  2002). 
Substantial  correlations  have  also  been  found  between  heart  rate  and  mental  workload, 
mental  capacity,  and  cognitive  situation  awareness.  Model  analyses  have  shown  a  mental 
workload  factor  that  contains  both  psychological  and  physiological  properties,  which  in 
turn  affects  cognitive  situation  awareness  (Svensson  &  Wilson,  2002).  As  mental 
workload  goes  up,  cognitive  situation  awareness  goes  down.  Consequently,  as  one 
measures  the  current  psychological  and  physiological  state  of  an  individual,  one  is 
measuring  mental  workload  and  in  turn,  cognitive  situation  awareness  (Gutzwiller  & 
Clegg,  2012).  This  previous  work  did  not,  however,  explore  visual  abilities  as  predictors 
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of  situation  awareness  or  faetors  between  mental  workload  and  situation  awareness 
(Kokar  &  Endsley,  2012;  Svensson  &  Wilson,  2002).  Predietive  models  of  eognitive 
situation  awareness  within  the  flight  deek  eontext  were  laeking  (Wiekens,  Sebok,  Keller, 
Peters,  Small,  Hutchins,  &  Foyle,  2013). 

Relating  Visual  Perception,  Neuropsychology,  and  SA 

The  human  eye  allows  one  to  detect  and  recognize  objects  that  produce  no  odor, 
have  no  taste  and  feel,  and  make  no  sounds.  Visual  perception  is  a  product  of  the  entire 
visual  system,  which  is  composed  of  the  eyes  and  several  areas  in  the  brain;  these  areas  in 
the  brain  are  vital  for  accurately  processing  visual  information.  The  eyes  serve  as 
transducers  that  receive  light  energy  and  convert  that  energy  into  electric  energy  by 
means  of  neuronal  processes  -  a  transduction  procedure  (Blake  &  Sekuler,  2006).  The 
world  around  us  involves  electromagnetic  energy  and  light  is  in  the  optical  spectrum  of 
that  electromagnetic  energy.  The  visual  system  permits  the  human  to  utilize  a  portion  of 
that  electromagnetic  energy  contained  in  the  visual  spectrum  by  converting  it  into  neural 
energy  (Blake  &  Sekuler,  2006).  This  neural  energy  is  processed  by  the  brain  so  that  the 
human  can  visually  perceive  what  is  out  there  in  the  world.  Visual  perception  is  both  a 
biological  and  psychological  process  and  seeing  is  a  very  dominant  sense  often 
overruling  the  other  four  senses.  Visible  light  is  electromagnetic  radiation  much  like 
radio  waves,  microwaves,  x-rays,  gamma  rays,  or  radar  waves;  together  they  make  up  the 
electromagnetic  (EM)  spectrum  (Blake  &  Sekuler,  2006). 

The  method  by  which  the  nervous  system  converts  stimuli  into  neural  events  is 
called  sensory  transduction  and  this  involves  neurons  at  the  most  fundamental  level.  The 
photoreceptors  in  the  eye  change  the  light  energy  entering  the  eye  into  electrical  energy 
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by  way  of  neurons  that  generate  neural  signals  (Blake  &  Sekuler,  2006).  The  pupil 
ehanges  size  to  regulate  how  much  light  enters  the  eye  and  reaches  the  retina.  All  human 
eyes  have  a  lens  that  assists  in  focusing  the  light  (Blake  &  Sekuler,  2006).  Near  the 
center  of  the  retina  is  where  vision  is  the  most  acute,  it  is  the  center  of  the  macula  and 
termed  the  fovea  (Blake  &  Sekuler,  2006).  At  the  rear  of  the  eye  is  the  optic  disk,  made 
up  of  optic  nerve  fibers  that  leave  the  retina  (Blake  &  Sekuler,  2006).  Transmitted, 
translucent,  and  reflected  light  from  objects  enters  the  human  eye  through  the  pupil,  it  is 
then  focused  on  the  retina  and  there  the  photoreceptors  biologically  convert  this 
information  into  electrical  signals  going  from  neuron  to  neuron  by  way  of  optic  nerve 
fibers  to  the  brain,  which  is  where  meaning  is  assigned  to  the  image  information. 

In  the  retina  the  last  stage  of  processing  -  determining  lightness,  sharpness, 
darkness,  contrast,  brightness,  occurs  in  the  neurons  called  retinal  ganglion  cells  (Blake 
&  Sekuler,  2006).  The  retinal  ganglion  cells  have  receptive  fields  that  are  composed  of 
ON  regions  and  OFF  regions  (Blake  &  Sekuler,  2006).  Naturally  ON  regions  respond  to 
increases  in  light  while  OFF  regions  respond  to  decreases  in  light.  A  stimulus  produces 
opposite  effects  in  the  center  then  the  surrounding  area  of  the  ON  and  OFF  regions  and 
therefore  the  retinal  ganglion  cells  function  via  an  antagonistic  process,  and  lateral 
inhibition  is  the  interaction  between  these  regions  (Blake  &  Sekuler,  2006).  The 
remaining  neurons  in  the  retina  are  composed  of  three  types:  Horizontal  cells.  Bipolar 
cells,  and  Amacrine  cells  (Blake  &  Sekuler,  2006).  Horizontal  cells  logically  connect 
photoreceptors  laterally  and  they  also  modulate  receptor  signals  (Blake  &  Sekuler,  2006). 
Bipolar  cells  receive  input  from  photoreceptors,  allowing  them  to  react  to  light  by 
changes  in  the  levels  of  glutamate;  more  light  causes  decreased  glutamate  and  decreases 
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in  light  causes  increases  in  glutamate,  and  therefore  the  bipolar  cells  produce  graded 
signals  corresponding  to  the  levels  of  light  (Blake  &  Sekuler,  2006).  Amacrine  cells 
modify  responses  of  the  bipolar  cells  acting  as  feedback  and  control  cells  (Blake  & 
Sekuler,  2006).  There  are  three  types  of  retinal  ganglion  cells:  Magnocellular  (M  cells), 
Parvocehular  (P  cells),  and  Koniocehular  (K  cells).  M,  P,  and  K  cells  differ  in  size, 
conductivity,  and  population,  and  allow  the  eye  to  produce  signals  for  spatial  resolution, 
temporal  resolution,  and  contrast  sensitivity,  respectively  (Blake  &  Sekuler,  2006). 

Each  optic  nerve  of  the  two  eyes  converges  at  the  optic  chiasm  (Blake  &  Sekuler, 
2006).  Contralateral  fibers  cross  to  the  opposite  side  of  the  brain  while  Ipsilateral  fibers 
from  each  eye  project  to  the  same  side  of  the  brain  (Blake  &  Sekuler,  2006).  These  optic 
tracts  run  from  the  chiasm  to  structures  deeper  into  the  brain  (Blake  &  Sekuler,  2006), 
including  the  lateral  geniculate  and  midbrain  structures,  though  some  fibers  project 
directly  to  the  cortex.  Information  from  the  optic  tracts  is  thus  transferred,  directly  and 
indirectly,  to  the  primary  visual  cortex  in  the  occipital  lobe.  Neural  information  from  the 
primary  visual  cortex  is  distributed  over  a  number  of  pathways  to  higher  visual  areas  of 
the  brain  (Blake  &  Sekuler,  2006).  Each  and  every  cortical  region  receiving  input  from 
another  region  also  sends  feedback  connections  back  to  that  other  region  creating  duality 
of  visual  processing  for  parallel  reliability  and  accuracy  (Blake  &  Sekuler,  2006). 

Visual  processing  is  an  important  part  of  SA  for  flying  an  aircraft  in  that  pilots 
scan  the  environment  to  include  the  IDs,  monitoring  the  aircraft’s  location  and  response 
to  his  or  her  actions  (Tirre  &  Gugerty,  1999,  2000).  A  component  of  performance  in 
dynamic  visual  environments  such  as  flying  is  temporal  processing.  Eor  example, 
temporal  processing  is  involved  in  estimating  aircraft  performance,  based  upon  control 
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inputs  that  are  decided  upon  based  on  PFI  presented  on  the  ID.  Visual  processes  that  are 
necessary  for  self-motion  (locomotion)  also  function  to  aid  the  pilot  in  landing  an 
airplane.  When  a  pilot  flies  the  airplane  on  final  approach,  his  or  her  vision  is  zeroing  in 
on  the  runway,  the  focus  is  on  the  X  intersection  of  the  1,000-foot  makers,  and  the  edges 
of  the  runway  have  an  outward  optic  flow,  if  indeed  the  motion  vector  of  the  aircraft  is 
centered  on  the  runway  (Gibson,  1986).  A  few  seconds  later,  when  the  pilot  is  flying  the 
airplane  in  the  landing  flare  and  is  about  to  touch-down,  they  have  to  judge  his  or  her 
time  to  contact  (airplane’s  wheels  time  of  impact  with  the  runway).  The  human  brain 
uses  internal  trigonometry  and  calculus  to  solve  the  Time  of  arrival  =  Distance  /  Rate 
problem  (where  the  symbol  7’  is  divided  by)  (Blake  &  Sekuler,  2006;  Stewart,  Redlin,  & 
Watson,  2007).  The  sight  transduction  process  and  intuitive  brain  calculation  procedure 
allows  the  human  visually  to  perceive  motion  in  the  world  and  maneuver  within  it. 
Attention  and  Motion  Perception 

Targets  are  detected  faster  when  their  spatial  location  is  cued  in  advance.  This  is 
why  flight  displays  in  cockpits  are  designed  in  a  way  with  appropriate  symbology  that 
will  cue  the  pilot  to  look  in  a  certain  direction  to  visually  acquire  the  target  faster, 
substantially  decreasing  target  acquisition  time.  Ball  and  Sekuler  (1981)  performed 
experiments  to  see  what  affects  attention  levels  would  have  on  motion  perception  (Blake 
&  Sekuler,  2006).  The  subjects  were  placed  in  front  of  a  video  screen  to  watch  dots 
appear  and  move  across  the  screen  from  various  directions,  and  at  different  light  levels  or 
no  dots  would  appear  at  all.  Ball  and  Sekuler  compared  the  intensity  thresholds  for 
subjects  when  they  were  not  given  any  precursor  cues  for  where  and  when  the  dots  would 
appear,  versus  when  the  subjects  were  given  cues.  They  found  that  when  subjects  were 
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cued  as  to  a  subsequent  dot  their  intensity  thresholds  for  deteetion  were  lower  than  when 
they  were  not  cued  (Blake  &  Sekuler,  2006).  The  three  top  findings  were:  (1)  If  the 
orientation  of  the  eue  preeisely  matehed  the  proeeeding  dot’s  direetion  of  motion,  the 
subjeets  had  the  lowest  threshold;  (2)  if  the  orientation  of  the  eue  only  simply 
approximated  the  proeeeding  dot’s  direetion  of  motion,  the  subjeets  had  a  medium 
threshold;  and  (3)  the  eue  was  not  helpful  unless  it  preeeded  the  dot  by  approximately  0.5 
seeonds,  giving  the  seleetive  attention  proeess  time  to  operate  (Blake  &  Sekuler,  2006). 
Having  the  cue  preeisely  mateh  the  proeeeding  dot’s  direetion  of  motion  in  some  ways 
illustrates  the  law  of  eommon  fate,  that  is  the  human’s  propensity  to  group  together 
individual  elements  that  are  moving  in  the  same  direetion  at  the  same  speed.  Potentially, 
that  may  be  the  best  way  to  design  the  most  effeetive  eue. 

Objective  of  the  Display 

The  main  purpose  of  the  ID  is  to  provide  the  pilot  informational  data  for  his  or  her 
SA  by  means  of  a  pictorial  display  containing  text  and  symbology.  If  the  ID  is  diffieult 
to  see,  eonfusing,  misleading,  or  unelear,  attention  eould  be  ehannelized  on  unimportant 
matters.  These  trivial  matters  eould  cause  pilots  to  miss  or  skip  vital  cheeklist  items 
pertaining  to  aircraft  configuration,  and  lose  eurrent  awareness  of  the  aireraft’s  six 
degrees  of  freedom  and  or  the  loeation  of  other  aireraft.  This  eould  ultimately  lead  to 
disastrous  results. 

Therefore,  this  researeh  also  examined  relationships  between  the  subject’s 
inherent  visual  attention,  visual-spatial  working  memory,  and  his  or  her  respeetive  level 
of  WL  and  SA  per  given  ID.  In  addition,  the  pereeptual  effeets  ereated  by  the  ID  on  these 
eognitive  phenomena  as  measured  by  the  level  of  SA  and  mental  WL  was  investigated. 
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Psychology  based  knowledge  about  the  relationships  between  pilot  neuroeognitive 
factors  and  SA  and  WL  will  be  signifieantly  inereased  as  will  knowledge  of  the 
pereeptions  ereated  as  a  funetion  of  ID  design  and  those  resulting  variables  (Bailey  et 
al.,2011;  Campbell,  2010;  Campbell  et  ah, 2010;  Crawford  and  Neal,  2006;  Flight  Safety 
Foundation,  2009;  Kim,  2009;  Kramer,  et  al  2005;  Wiekens,  2008). 

Display  Design 

The  pilot  does  not  have  to  switeh  attention  baek  and  forth  between  the  HUD  and 
the  outside  world  when  the  the  HUD  is  easily  visible  to  the  pilot  while  in  a  normal 
seating  position  and  aligned  with  the  outside  view.  Therefore,  the  HUD  should  enhanee  a 
pilot’s  ability  to  detect  events  in  the  external  world  as  opposed  to  having  to  task  switch 
between  a  traditional  heads  down  (instrument  panel)  display  and  the  external 
environment  (Parasuraman  and  Rizzo,  2008). 

The  input  proeess  for  SA  formation  is  aeeomplished  by  way  of  information 
proeessing,  whieh  is  critieally  dependent  on  attention  (Crawford,  2006).  Consequently, 
the  pilot’s  inherent  visual  attention  trait  plays  an  important  role  in  this  proeess. 
Demonstrated  by  researeh,  in  general,  humans  are  mueh  better  at  deteeting  events  in  the 
environment  if  his  or  her  attention  is  foeused  on  the  area  in  whieh  those  events  oecur 
(Wiekens  &  Hollands,  2000).  However,  attention  is  a  limited  capaeity  resouree.  An  ID 
design  that  eneourages  divided  attention  may  inhibit  the  pilot’s  ability  to  foeus  attention 
on  speeifie  aspects  of  the  ID  (Verver  s  &  Wiekens,  1998).  For  example,  if  the  pilot  can 
only  see  a  portion  of  the  symbology  on  the  HUD,  they  will  then  need  to  divide  attention 
and  eross  eheek  the  HDD.  For  this  reason,  the  HUD  and  the  external  world  may  not  be 
proeessed  or  visually  attended  to  at  the  same  time  as  was  intended.  This  results  in  a 
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latency  of  information  being  processed  because  of  the  unattended  domain  being 
perceived  only  after  some  delay,  therefore,  change  detection  may  be  degraded  (Crawford, 
2006;  McCann,  Lynch,  Foyle,  &  Johnston,  1993;  Moodi,  1995).  Perceptual  aspects  of 
the  HUD  visual  field  should  be  studied  further  (Crawford,  2006).  The  HUD  should  be 
designed  to  improve  the  ability  of  the  pilot  to  concentrate  attention  robustly  on  the 
outside  world  (Crawford,  2006). 

In  a  fixed-based  simulator,  an  initial  study  comparing  HUDs  with  HDDs  found 
that  landings  were  more  accurate  using  HUDs  (Crawford,  2006).  High  workload  is 
associated  with  an  increase  in  cognitive  tunneling  or  focusing  on  one  item  or  event 
causing  other  items  of  importance  to  be  missed  (Dowell,  Foyle,  Hooey,  and  Williams, 
2002).  It  is  believed  that  cognitive  tunneling  is  caused  by  limitations  in  attentional 
capacity;  therefore,  increasing  WL  would  further  reduce  a  pilot’s  available  capacity,  in  so 
doing  worsening  the  tunneling  effect  (Crawford,  2006).  However,  there  are  very  few 
studies  comparing  the  workload  of  HUD  and  HDD  (Crawford  &  Neal,  2006). 

The  work  performed  by  Yamani  and  McCarley  (2011),  accomplished  experiments 
to  test  whether  design  of  symbology  to  produce  visual  search  asymmetries  could  facilitate 
target  detection  in  cluttered  displays.  Visual  search  asymmetry  exists  when,  in  two 
stimuli  a  target  of  one  type  is  found  efficiently  among  distractors  of  the  second  type, 
conversely  a  target  of  the  second  type  would  be  found  with  difficulty  among  distractors 
of  the  first  type.  In  this  study,  ten  participants  performed  a  visual  search  task  using 
stimuli  (canonical  vs.  reversed  Ns)  known  to  produce  a  search  asymmetry.  The  search 
stimuli  were  either  embedded  within  images  containing  either  low  or  high  levels  of 
clutter.  For  statistical  analysis,  the  values  were  submitted  to  a  2  x  2  x  5  within-subject 
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ANOVA  with  clutter  (high  vs.  low),  target  type  (N  vs.  reversed  N),  and  exposure  duration 
(250,  750,  1,500,  2,500,  and  5,000  ms)  as  factors.  The  background  images  with  heavy 
clutter  produced  lower  sensitivity  than  those  with  less  clutter,  F(i,9)  =  78.46, <.0\,MSE 
=  .03,  ri2  p  =  .90,  and  sensitivity  was  higher  when  the  target  was  a  reverse  N  among 
canonical  N  distractors  than  when  the  opposite  target-distraetor  mapping  was  used,  F(i,  9) 
=  104.89, />  <  .01,  =  .003,  r|2  p  =  .92,  demonstrating  the  expected  search  asymmetry 

(Yamani  &  McCarley,  2011).  Displays  with  heavy  clutter  required  more  processing 
time  than  less  cluttered  displays,  F(i,9)  =  48.84, <  .01,  MSE  =  273165.28,  ri2  p  =  .84, 
and  mean  processing  time  for  the  reverse-A  targets  was  significantly  shorter  than  for  the 
eanonical-A  targets,  F(i,  9)  =  1.51,p  =  .02,  MSE  =  400181.01,  r|2  p  =  .46,  demonstrating  a 
seareh  asymmetry.  Illustrating  that  heavy  clutter  did  not  reduce  the  strength  of  the 
A/reversed-A  asymmetry  in  processing  rates.  The  results  showed  that  the  search 
asymmetry  was  robust  against  the  presence  of  heavy  display  clutter.  Therefore,  seareh 
asymmetries  are  robust  against  heavy,  spatially  continuous  visual  clutter,  and  this  could 
be  utilized  to  improve  design  of  display  symbology  to  maximize  detectability  of  task- 
critical  information  in  complex  displays. 

Cognitive  Modeling 

The  study.  Using  Computational  Cognitive  Modeling  to  Diagnose  Possible 
Sourees  of  Aviation  Error  (Byrne  &  Kirlik,  2005),  sought  to  expand  knowledge  in 
psyehology  in  a  way  that  allows  the  use  of  Artifieial  Intelligence  (AI)  to  diagnose  the 
causes  of  aviation  error.  This  adds  to  theory  in  psyehology  in  two  ways:  by  finding  the 
potential  root  causes  of  pilot  error  and  by  finding  why  and  how  the  AI  program  is  able,  or 
not  able,  to  model  human  behavior  in  order  to  discover  the  eauses  of  this  error.  The 
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study  utilized  the  Adaptive  Control  of  Thought-Rational  (ACT-R)  eognitive  modeling/ AI 
arehiteeture.  For  the  eognitive  aspects  in  ACT-R,  Subject  Matter  Experts  (SME)  were 
used  to  identify  the  pragmatic  adaptations  pilots  bring  to  taxiing  an  aircraft  (Byrne  & 
Kirlik,  2005).  Next,  the  SME  information  was  placed  into  the  ACT-R  computational 
model.  This  model  was  closed-loop  with  the  following  links:  pilot-aircraft-visual  scene¬ 
taxiway.  This  system  was  created  to  potentially  identify  sources  of  taxi  error. 

The  distinct  contribution  to  the  expansion  of  psychological  knowledge  was  that 
the  derived  five  decision  strategies  in  an  ACT-R  model,  ranging  from  cognitively 
demanding,  but  precise,  to  fast  and  careful,  but  robust,  was  accurate  (Byrne  &  Kirlik, 
2005).  When  the  ACT-R  model  was  compared  by  its  behavior  to  a  National  Aeronautics 
and  Space  Administration  (NASA)  Ames  Research  Center  simulation  of  Chicago  O’Hare 
surface  operations,  the  model  selected  the  most  accurate  strategy  even  though  the 
decision  horizons  were  highly  variable  (Byrne  &  Kirlik,  2005).  The  contributions  to 
theory  were  that  a  common  thread  in  the  simulation  data  revealed  that  errors  occurred 
most  frequently  at  atypical  taxiway  geometries  or  clearance  routes,  and  the  data  provided 
empirical  support  for  this  ACT-R  model  (Byrne  &  Kirlik,  2005).  In  addition,  the 
accurately  predicted  human  behavior  was  due  to  being  able  to  cope  with  short  decision 
horizons  by  the  use  of  globally  robust  heuristics  in  the  ACT-R  program  (Byrne  &  Kirlik, 
2005). 

The  NASA’s  Synthetic  Vision  Systems  (SVS)  project  is  focused  on  developing 
technologies  that  will  replicate  clear  day  flight  operations,  regardless  of  the  actual  outside 
visibility  condition  thereby  eliminating  low  visibility  conditions  as  a  causal  factor  to 
aircraft  accidents  (Kramer,  Prinzel,  Arthur,  &  Bailey,  2004).  This  study  expanded 
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psychological  knowledge  by  assessing  the  effeet  of  different  guidanee  eoneepts  upon 
pilot  Situation  Awareness  (SA),  mental  Work  Load  (WL),  and  aircraft  flight  path 
tracking  performance  for  an  SVS  system  using  a  Head-Up  Display  (HUD)  (Kramer  et 
al,  2004). 

Speeifieally,  two  eentral  guidance  concepts  were  assessed  using  tunnel  formats: 
dynamie  or  minimal,  and  they  were  also  assessed  against  the  baseline  condition  (no 
tunnel)  (Kramer  et  ah,  2004).  This  was  accomplished  in  a  flight  simulator  with  simulated 
Instrument  Meteorological  Conditions  (IMC)  flying  approaches  to  Reno-Tahoe 
International  airport  (Kramer  et  ah,  2004).  Two  main  guidanee  cues,  tadpole  and  follow- 
me  aircraft,  were  also  evaluated  in  the  both  the  dynamic  and  minimal  guidance  domains 
to  assess  their  effect  with  tunnel  formats  (Kramer  et  ah,  2004). 

The  particular  psychology  knowledge  gained  was  that  the  presence  of  a  tunnel  on 
an  SVS  HUD  had  no  effeet  on  flight  path  performanee.  However,  it  did  have  significant 
effects  on  pilot  SA  and  WL  (Kramer  et  ah,  2004).  The  psyehology  theory  that  was  gained 
by  this  study  is  that  the  relationships  among  the  variables  of  different  SVS  HUD 
guidance  concepts  and  SA  and  WL  is  that  the  dynamic  tunnel  eoneept  with  the  follow-me 
aircraft  guidance  symbol  ereated  the  lowest  workload  for  the  pilot  while  providing  the 
highest  level  of  SA  among  the  concepts  assessed  (Kramer  et  ah,  2004). 

For  the  next  study  using  a  densely  populated  battlefield  Current  Operational 
Pieture  (COP),  different  icon  modalities  were  assessed  to  evaluate  the  effects  on  operator 
Situation  Awareness  (SA)  (Strater,  Riley,  Faulkner,  Hyatt,  &  Endsley,  2006).  For  a 
baseline,  military  standard  battle  field  symbology  was  compared  to  various  modified 
versions  of  this  symbology.  The  Situation  Awareness  Global  Assessment  Teehnique 
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(SAGAT)  was  the  instrument  utilized  to  extract  SA  data  after  each  exercise  using  the 
different  symbology.  Subjects  also  indicated  their  perceived  utility  of  the  icon  being 
assessed  in  terms  of  supporting  SA  as  part  of  data  collection. 

The  study’s  contribution  to  increasing  psychology  knowledge  is  that,  as  showed 
by  the  objective  SAGAT  results,  subject  SA  performance  was  greatest  using  the 
proportional  icon  modality  (Strater  et  ah,  2006).  It  should  be  noted  that  the  subjects’ 
perception  which  was  strongly  influenced  based  upon  what  they  were  the  most  familiar 
with,  was  that  the  current  military  standard  representations  are  easiest  for  assisting  them 
in  monitoring  friendly  and  enemy  forces.  However,  as  indicated  above,  the  objective 
SAGAT  measure  showed  different  (Strater  et  ah,  2006)  results.  The  study’s  contribution 
to  psychology  theory  was  twofold:  1)  it  illustrated  the  relationship  between  the  variables 
of  icon  modalities  and  SA  and  2)  the  findings  also  verified  that  a  relatively  minor  design 
difference  in  icon  pattern  can  have  a  significant  impact  on  operator  SA  (Strater  et  ah, 
2006).  Increasing  SA  increases  the  available  cognitive  capacity  that  can  be  directed 
toward  other  tasks  (Strater  et  ah,  2006).  It  also  showed  once  again  how  objective  and 
subjective  measurement  could  diverge. 

Summary 

Situation  awareness,  as  codified  in  the  TMSA  and  applied  to  the  task  of  piloting 
an  aircraft,  is  built  upon  fundamental  cognitive  abilities.  These  include  visual  attention, 
as  detailed  in  the  SEEV  model,  and  spatial  working  memory,  as  discussed  in  Baddeley’s 
work.  Mental  workload  is  a  critical  construct,  as  overloading  the  resources  identified  in 
the  MRT  will  cause  a  considerable  degradation  of  situation  awareness.  There  was  no 
consensus  on  predictors  for  situation  awareness,  and  this  area  tended  to  be  elusive.  None 
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of  the  extant  work  had  thus  far  successfully  explored  the  individual  differences  in  visual 
abilities  that  are  critical  to  forming  and  maintaining  SA  (Douglas,  Aleva,  &  Havig,  2007; 
Ellis  &  Levy,  2009;  Elliott  et  ah,  2009;  Endsley,  2012;  Gillan  et  ah,  2009;  Gugerty,  in 
press;  Gutzwiller  &  Clegg,  2012;  Harbour  et  ah,  2012;  Jen-li,  Ruey-Yun,  &  Ching-Jung, 
2013;  Jones,  Connors,  &  Endsley,  2011;  Jodlowski,  2009;  Lau,  Jamieson,  &  Skranning, 
2013;  Proctor  &  Vu,  2010;  Sulistyawati,  Wickens,  &  Chui,  2011;  Vidulich  &  Tsang, 

2012;  Wickens,  2008;  Wickens  &  McCarley,  2008;  Tirre  &  Gugerty,  1999,  2000)  and 
consequently  this  dissertation  is  the  logical  next  step  in  that  scientific  endeavor,  through 
inquiry  and  discovery,  towards  theory  in  psychology.  This  study  will  improve  the  TMSA 
by  finding  the  relationships  between  inherent  neurocognitive  factors  and  changes  in 
situation  awareness  and  workload,  a  critical,  substantial,  and  significant  gap  in  the 
TMSA. 

These  relationships  were  explored  by  varying  task  difficulty,  induced  via 
manipulation  of  the  ID  spatial  location  and  usability.  This  research  dissertation  sought  to 
maximize  ecological  validity  by  conducting  data  collection  during  actual  flight,  and 
maximized  the  knowledge  gained  through  the  application  of  neuro ergonomic  techniques. 
Irrespective  of  the  scientific  necessity,  there  was  no  firm  grounding  for  current  theories  of 
situation  awareness  in  fundamental  cognitive  and  perceptual  processes  as  links  and 
predictors.  Since  Endsley’s  Theoretical  Model  of  Situation  Awareness  (TMSA,  1995a) 
was  first  published,  models  of  situation  awareness  were  still  conceptual  models  that 
provided  little  specificity  with  regards  to  the  neurocognitive  processes  that  are  necessary 
for  the  formation  and  maintenance  of  situation  awareness  (Gillan  et  al.,  2009;  Lau, 
Jamieson,  &  Skranning,  2013;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley, 
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2008).  While  the  TMSA  recognized  perception  as  a  critical  first  step,  there  were  no 
specific  or  quantitative  links  between  perceptual  abilities  and  situation  awareness;  nor 
had  subsequent  work  been  able  to  clarify  the  issue,  e.g.  the  effects  of  visual  ability  on  the 
level  of  situation  awareness  (Jen-li,  Ruey-Yun,  &  Ching-Jung,  2013;  Jones,  Connors,  & 
Endsley,  2011;  Vidulich  &  Tsang,  2012;  Sulistyawati,  Wickens,  &  Chui,  2011).  Needed 
theoretical  advancement  in  this  area  continued  to  be  hampered  by  a  lack  of  specific, 
testable  predictions  regarding  plausible  component  processes;  there  had  been  little 
theoretical  progress  due  to  this  (Douglas,  Aleva,  &  Havig,  2007;  Endsley,  2012; 
Gutzwiller  &  Clegg,  2012;  Harbour  et  ah,  2012;  Vidulich  &  Tsang,  2012;  Wickens, 

2008;  Sulistyawati,  Wickens,  &  Chui,  2011). 

The  seminal  research  done  by  Tirre  and  Gugerty  (1999,  2000),  Wickens  and 
McCarley  (2008),  Elliott  et  ah,  (2009),  Jen-li,  et  ah,  (2013),  and  Gugerty,  (in  press)  found 
that  visual  processing  (Proctor  &  Vu,  2010)  was  involved  in  situation  awareness 
(Vidulich  &  Tsang,  2012).  However,  the  specific  component  processes  were  yet  to  be 
explored  in  detail  (Gutzwiller  &  Clegg,  2012;  Gugerty,  in  press;  Harbour  et  ah,  2012; 
Jen-li  et  ah,  2013;  Jones  &  Endsley,  2012;  Sulistyawati  et  ah,  2011;  Vidulich  &  Tsang, 
2012).  Visual  processing  can  be  categorized  as  static  and  dynamic  (Proctor  &  Vu,  2010). 

As  situation  awareness  develops,  in  the  same  way,  as  discourse  comprehension, 
event-base  development  begins  through  a  strictly  bottom-up  process  driven  by  sensory 
information.  Therefore,  for  Eevel  1  in  situation  awareness  (perception)  to  occur  correctly 
it  is  vital  to  the  initial  stage  of  visual  perception  and  the  processing  of  that  information  to 
also  occur  properly.  Top-down  processing  is  utilized  to  allow  assigning  the  appropriate 
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meaning  in  a  particular  context  to  the  visual  information,  which  requires  WM, 
specifically  visual-spatial  WM  (Durso  et  ah,  2007;  Matlin,  2008)  for  spatial  information. 

This  would  entail  three  variables:  visual  perceptiveness,  visual  attentiveness,  and 
visual-spatial  working  memory  (WM)  (Blake  &  Sekuler,  2006;  Brain  Train,  2010; 

Matlin,  2008;  Carlson,  2010  ).  These  maybe  summarized  as:  a)  did  the  pilot  see  it  at  all, 
b)  did  the  pilot  attend  to  it,  and  c)  did  the  pilot  encode  it  in  memory  such  that  he  or  she 
could  use  it  in  building  a  situation  model.  An  understanding  of  visual  processing  factors 
(such  as  independent  variable/s)  operating  in  dynamic  environments  was  incomplete 
(Crawford  &  Neal,  2006;  Elliott  et  ah,  2009;  Tirre  &  Gugerty,  1999;  Wickens,  2002). 

This  suggested  that  the  inherent  neurocognitive  factors  of  visual  attention  (perceptiveness 
and  attentiveness)  and  visual-spatial  working  memory,  as  part  of  a  theory  may  explain 
their  support  for  the  development  of  situation  awareness.  Consequently,  these  may  be 
operationalized  as  visual  attentiveness  and  perceptiveness,  integrated  with  visuospatial 
memory  (Brain  Train,  2010;  Christensen  et  ah,  2013;  Corbett  &  Constantine,  2007; 
Endsley,  2012;  Gugerty,  in  press;  Sandford,  Eine,  &  Goldman,  1995;  Sandford  &  Turner, 
1994;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley,  2008). 

Therefore,  visual  attention  (perceptiveness  and  attentiveness)  and  visual-spatial 
working  memory  and  processing  were  treated  as  independent  variables  in  order  to  better 
develop  a  definitive  psychological  model  and  theory  for  situation  awareness  prediction 
and  understanding,  known  as  the  Enhanced-TMSA.  The  current  study,  therefore,  tested 
explicit  hypotheses  regarding  specific  abilities  that  contribute  to  situation  awareness;  the 
results  of  which  fill  in  a  critical  gap  in  the  TMSA,  and  in  so  doing  enable  both  theoretical 
refinement  (providing  an  Enhanced-Theoretical  Model  of  SA)  and  practical  applications 
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such  as  improved  procedures,  training  for  pilots,  and  display  design  that  improve  flight 
safety. 
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Chapter  3:  Research  Method 

Introduction 

The  experimental  study  of  situation  awareness  presents  particular  challenges. 
Situation  awareness  is  highly  context-specific;  experimental  testing  and  manipulation 
must  be  conducted  with  respect  to  an  actual  task.  As  previously  discussed,  it  is  a 
complicated  tangible  that  is  not  well  addressed  by  laboratory  study  focusing  on  simple, 
artificial  tasks.  Consequently,  the  use  of  a  meaningful,  consequential  task  context  is 
essential  to  the  study  of  situation  awareness.  This  chapter  will  review  the  problem  and 
purpose  of  the  study,  and  then  link  the  study’s  hypotheses  to  the  experimental  design 
with  a  real-world  task  that  tested  those  hypotheses. 

Despite  the  scientific  necessity,  there  was  no  firm  grounding  for  current  theories 
of  situation  awareness  in  fundamental  cognitive  and  perceptual  processes  still.  In  the 
face  of  nearly  20  years  of  work  since  Endsley’s  Theoretical  Model  of  Situation 
Awareness  (TMSA,  1995a)  was  first  published,  current  models  of  situation  awareness 
were  yet  conceptual  models  that  provided  little  specificity  with  regards  to  the 
neurocognitive  processes  that  are  necessary  for  the  formation  and  maintenance  of 
situation  awareness  (Gillan  et  ah,  2009;  Lau,  Jamieson,  &  Skranning,  2013;  Vidulich  & 
Tsang,  2012;  Wickens  &  McCarley,  2008).  While  the  TMSA  recognized  perception  as  a 
critical  first  step,  there  were  no  specific  or  quantitative  links  between  perceptual  abilities 
and  situation  awareness;  nor  has  repeated  subsequent  work  been  able  to  elucidate  the 
issue  (Jen-li,  Ruey-Yun,  &  Ching-Jung,  2013;  Jones,  Connors,  &  Endsley,  2011; 
Vidulich  &  Tsang,  2012;  Sulistyawati,  Wickens,  &  Chui,  2011).  Needed  theoretical 
advancement  in  this  area  continued  to  be  hampered  by  a  lack  of  specific,  testable 
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predictions  regarding  plausible  component  processes;  there  had  been  little  theoretical 
progress  because  of  this  (Douglas,  Aleva,  &  Havig,  2007;  Endsley,  2012;  Gutzwiller  & 
Clegg,  2012;  Harbour  et  ah,  2012;  Vidulich  &  Tsang,  2012;  Wickens,  2008;  Sulistyawati, 
Wickens,  &  Chui,  2011). 

The  seminal  research  done  by  Tirre  and  Gugerty  (1999,  2000),  Wickens  and 
McCarley  (2008),  Elliott  et  ah,  (2009),  Jen-li,  et  ah,  (2013),  and  Gugerty  (in  press)  found 
that  visual  processing  (Proctor  &  Vu,  2010)  was  involved  in  situation  awareness 
(Vidulich  &  Tsang,  2012).  However,  the  specific  component  processes  were  not  yet 
explored  in  detail  (Gutzwiller  &  Clegg,  2012;  Gugerty,  in  press;  Harbour  et  ah,  2012; 
Jen-li  et  ah,  2013;  Jones  &  Endsley,  2012;  Sulistyawati  et  ah,  2011;  Vidulich  &  Tsang, 
2012).  Therefore  this  work  tested  explicit  hypotheses  regarding  specific  abilities  that 
contribute  to  situation  awareness;  the  results  of  this  study  fills  in  a  critical  gap  in  the 
TMSA,  and  in  so  doing  enables  both  theoretical  refinement  and  practical  applications 
such  as  improved  procedures,  training  for  pilots,  and  display  design  that  improve  flight 
safety. 

Visual  processing  can  be  categorized  as  static  and  dynamic  (Proctor  &  Vu,  2010), 
and  may  be  operationalized  as  visual  attentiveness  and  perceptiveness,  integrated  with 
visuospatial  memory  (Brain  Train,  2010;  Christensen  et  ah,  2013;  Corbett  &  Constantine, 
2007;  Endsley,  2012;  Gugerty,  in  press;  Sandford,  Eine,  &  Goldman,  1995;  Sandford  & 
Turner,  1994;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley,  2008).  Consequently,  the 
primary  purpose  of  this  quasi-experimental  quantitative  study  was  to  test  the  predictive 
value  of  these  specific  candidate  visuo-cognitive  abilities  as  predictors  of  situation 
awareness.  Therefore,  this  project  tested  the  predictive  value  of  these  variables  as  factors 
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between  a  particular  task  to  be  performed  and  the  eventual  outcome  of  situation 
awareness.  As  Pavlov,  Watson,  and  Skinner  performed  field  experiments  in  order  to 
contribute  to  psychological  theory  (Cervone  &  Pervin,  2007),  a  field  experiment  in  the 
paradigm  of  neuroergonomics  (Parasuraman,  Christensen,  &  Grafton,  2012)  employing 
quasi-experimental  repeated-measures  (within-participants),  was  used  for  this  study. 

This  was  the  most  effective  and  realistic  method  to  perform  the  research  that 
addressed  the  problem  and  the  purpose.  As  computed  by  G*power,  this  study  required  a 
sample  size  of  a  minimum  of  12  United  States  Air  Force  (USAF)  pilots.  These  pilots 
were  recruited  at  an  airport  in  north  central  Ohio  by  utilizing  a  recruitment  memo  in 
squadron  buildings.  All  participants  were  pilots  qualified  in  the  aircraft.  There  were  two 
display  conditions  that  varied  the  difficulty  of  maintaining  SA  by  varying  the  presence 
and  position  of  the  information  display.  The  first  condition  had  an  Information  Display 
(ID)  design  that  was  near  optimal  and  centered  on  the  pilot’s  field  of  view  and  was  no 
more  than  five  degrees  above  or  below  the  pilot’s  line  of  sight  (Harbour,  Hudson,  & 
Zehner,  2012).  The  second  condition  had  an  ID  design  that  was  suboptimal  and  well  out 
of  pilot  line-of-sight  (a  35  degree  vertical  drop)  (Harbour,  Hudson,  &  Zehner,  2012). 

This  display  manipulation  induced  variation  in  task  difficulty,  to  observe  differential 
effects  on  both  subjective  and  psychophysiological  measures  (see  Figure  3).  This  work 
was  successful  and  will  result  in  significant  elaboration  of  theoretical  models  of  situation 
awareness  as  well  as  enabling  and  focusing  efforts  to  improve  situation  awareness.  The 
bottom  line  is  what  were  the  effects  of  visual  ability  on  the  level  of  situation  awareness? 
To  address  the  research  questions,  the  following  hypotheses  were  tested: 
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Ql.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  attentiveness  on  situation  awareness? 

Q2.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  perceptiveness  on  situation  awareness? 

Q3.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  spatial  working  memory  on  situation  awareness? 
Hypotheses 

To  address  the  research  questions,  the  following  hypotheses  were  tested: 

Hlo.  Under  varying  task  difficulty,  there  was  no  statistically  significant  effects  of 
inherent  visual  attentiveness,  (as  measured  by  the  Integrated  Visual  and  Auditory 
Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on  resulting 
situation  awareness. 

Hla.  Under  varying  task  difficulty,  there  was  statistically  significant  effects  of 
inherent  visual  attentiveness,  (as  measured  by  the  Integrated  Visual  and  Auditory 
Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on  resulting 
situation  awareness. 

H2o.  Under  varying  task  difficulty,  there  was  no  statistically  significant  effects  of 
inherent  visual  perceptiveness,  (as  measured  by  the  Integrated  Visual  and 
Auditory  Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on 
resulting  situation  awareness. 

H2a.  Under  varying  task  difficulty,  there  was  statistically  significant  effects  of 
inherent  visual  perceptiveness,  (as  measured  by  the  Integrated  Visual  and 
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Auditory  Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on 
resulting  situation  awareness. 

H3o.  Under  varying  task  difficulty,  there  was  no  statistically  significant  effects  of 
Visuospatial  Working  Memory  Test  results  (AFRL,  2009),  on  resulting  situation 
awareness. 

H3a.  Under  varying  task  difficulty,  there  was  statistically  significant  effects  of 
Visuospatial  Working  Memory  Test  results  (AFRL,  2009),  on  resulting  situation 
awareness. 

Research  Methods  and  Design 

The  primary  purpose  of  this  quasi-experimental  quantitative  study  was  to  test 
visual  perceptiveness,  attentiveness,  and  spatial  working  memory  as  predictors  of 
situation  awareness.  In  order  to  maximize  ecological  validity  and  minimize  nuisance 
variance  due  to  heterogeneous  participants,  this  project  focused  on  the  controlled 
assessment  of  the  visuocognitive  abilities  of  Air  National  Guard  pilots,  with  actual 
piloting  as  a  completely  real  task  suitable  for  measuring  resulting  situation  awareness. 

This  was  the  most  effective  and  realistic  method  to  perform  the  research  that 
addresses  the  problem  and  the  purpose.  As  an  example  alternative  approach,  a  wholly 
laboratory  based  study  utilizing  the  typical  convenience  sample  of  college  undergraduates 
could  have  been  accomplished;  however,  one  would  expect  substantial  individual 
differences  as  well  as  serious  concerns  about  the  measurement  of  situation  awareness 
outside  any  meaningful  task  context. 

In  order  to  induce  meaningful  variation  in  the  outcome  measures  of  situation 
awareness,  task  difficulty  in  the  flight  task  was  manipulated  between  two  levels:  a  low 
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difficulty  level  associated  with  effective  and  easy-to-use  information  displays  and  a  high 
difficulty  level  associated  with  ineffective  information  displays.  Each  pilot  was  assessed 
for  the  three  candidate  visual  abilities  (attentiveness,  perceptiveness,  and  visuospatial 
working  memory).  This  approach  resulted  in  a  quasi-experimental  design  that  made  use 
of  repeated  measures  comparisons  (Cozby,  2001;  see  Table  3)  four  times  per  subject,  in 
order  to  achieve  required  statistical  power. 

The  experiment  was  organized  in  much  the  same  manner  as  a  strict  experimental 
design,  however,  due  to  the  operational  realism  involved,  it  lacked  complete  randomness 
(Creswell,  2009;  de  Vans,  2001;  Moore,  2007).  This  lack  of  randomness  was  recognized 
and  was  accounted  for  during  the  entire  experimental  process  (Trochim  &  Donnelly, 
2008).  This  tradeoff  between  experimental  control  and  ecological  validity,  while  a 
compromise,  improved  the  validity,  generalizability,  and  significance  of  this  study 
(Alasuutari,  Bickman,  &  Brannen,  2008). 

Correlation  and  causation  analysis  utilizes  mathematical  tools  such  as  the  within- 
subjects  and  factors  repeated-measures  Analysis  of  Variance  (ANOVA)  and  linear 
multiple  regression  modeling  to  study  effects  on  one  dependent  variable  by  more  than 
one  independent  variable  (Grim  &  Yarnold,  2006).  The  factorial  Analysis  of  Variance 
(ANOVA)  begins  with  the  premise  of  the  foundation  of  multivariate  statistics  in 
providing  a  simultaneous  analysis  of  multiple  independent  variables  and  the  dependent 


variable  (Grim  &  Yarnold,  2006). 
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Table  3 

The  2X3  Matrix 


Visual  Abilities  (IVs) 

Va 

Vp  V  swm 

Display  Condition 

HUD  SA 

SA  SA 

(Task  Difficulty) 

HDD  SA 

SA  SA 

Table  3  illustrates  that  task  diffieulty  was  varied  by  display  use  manipulation  (a 
eontrolled  eovariate  variable)  so  the  effeets  of  visual  abilities  (Va,  Vp,  and  Vswm)  on  the 
level  of  SA  eould  be  more  realistically  studied. 

Multiple  ANOVAs  would  be  required  to  test  for  each  independent  variable; 
therefore,  the  factorial  ANOVA  is  preferred;  this  is  an  ANOVA  with  several  independent 
variables  (Grim  &  Yarnold,  2006).  The  factorial  ANOVA  has  advantages  over  the 
ANOVA.  There  is  an  increased  chance  of  discovering  which  factor  is  most  important,  by 
measuring  several  independent  variables  in  the  same  experiment  (Tabachnick,  &  Fidell, 
1996).  Differences  in  variables  can  be  more  salient  utilizing  the  factorial  ANOVA  versus 
using  several  ANOVAs.  The  factorial  ANOVA  also  allows  for  better  control  of  statistical 
error  in  significance  testing.  There  are  two  kinds  of  errors  that  can  be  made  in 
significance  testing:  type  I  and  type  II.  Type  I  is  considered  to  be  the  most  serious  and  it 
is  when  a  true  null  hypothesis  is  incorrectly  rejected.  Type  I  error  can  occur  if  multiple 
ANOVA’s  are  conducted  independently,  whereas  using  a  single  factorial  ANOVA 
instead  could  reduce  the  likelihood  of  Type  I  error  (Green  &  Salkind,  2008;  Grim  & 
Yarnold,  2006;  Tabachnick,  &  Fidell,  1996). 

This  ANOVA  type  has  the  ability  to  illustrate  the  effects  of  the  input  (independent 
variables)  on  the  output  (dependent  variable)  of  the  system  in  the  experiment,  while 
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reducing  the  likelihood  of  type  I  error.  The  within-subjects  and  factors  repeated- 
measures  ANOVA  was  used  to  measure  the  main  effects  of  the  independent  variables 
(task  difficulty  and  three  visual  abilities)  the  interactions  among  the  independent 
variables,  the  importance  of  the  dependent  variable  (situation  awareness),  and  the 
strength  of  association  between  these  variables. 

This  experiment/research  was  a  within-subject  design.  In  between-subject 
designs,  the  differences  among  participants  are  uncontrolled  and  are  treated  as  error 
(Jackson,  2009;  Shavelson,  1996).  In  within-subject  designs,  the  same  participants  are 
tested  in  each  condition,  with  repeated  sampling  of  each  condition.  Therefore, 
differences  among  participants  can  be  measured  and  separated  from  error  (Jackson,  2009; 
Shavelson,  1996).  Removing  variance  due  to  differences  between  participants  from  the 
error  variance  substantially  increases  the  power  of  significance  tests  (Grim  &  Yarnold, 
2006).  Therefore,  within-participants  designs  are  almost  always  more  powerful  than 
between-subject  designs.  Since  power  is  such  an  important  consideration  in  the  design  of 
experiments,  within-subject  designs  are  generally  preferable  to  between-subject  designs 
(Jackson,  2009;  Shavelson,  1996). 

Population 

For  this  study,  the  population  was  Ohio  Air  National  Guard  pilots  (part  of  the 
U.S.  Air  Force),  which  are  a  relatively  homogeneous  group;  the  Air  Force  in  a  rigorous 
process  selects  these  individuals,  and  all  have  completed  highly  regimented  initial  and 
recurrent  training.  Consequently,  this  homogenous,  expert  population  of  26  (USAF, 
2013)  are  more  likely  to  show  consistent  effects  of  both  task  difficulty  and  visual 
abilities.  The  differences  are  and  were  in  the  pilots’  level  of  neurocognitive  ability. 
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because  while  visual  acuity  is  carefully  controlled,  the  USAF  currently  has  no  established 
requirements  or  tests  for  the  neurocognitive  visual  abilities  (Butcher,  2007). 

Sample 

Participants  were  recruited  from  the  National  Guard,  by  circulating  memos 
throughout  the  base.  This  targeted  sampling  is  necessary,  as  the  study  requires  a 
relatively  homogenous,  expert  population  from  both  a  real  world  and  scientific 
standpoint.  On  the  practical  side,  participants  must  be  qualified  and  experienced  Ohio 
Air  National  Guard  pilots  in  order  to  fly  the  aircraft.  From  a  scientific  perspective,  this 
relatively  homogenous  participant  pool  helps  ensure  that  observed  differences  are 
traceable  to  the  study  manipulations,  and  do  not  reflect  error  differences.  This  does  have 
the  consequence  of  reducing  the  ability  to  generalize  to  non-studied  populations,  but  the 
tradeoff  was  felt  acceptable  in  light  of  the  many  other  benefits  of  the  study  design, 
including  high  ecological  validity  and  significantly  reduced  complexity  and  cost  in  the 
conduct  of  data  collection.  All  participants  were  qualified  pilots  in  the  aircraft  that  was 
used  for  the  study.  There  were  no  restrictions  as  to  geographical  location  or  ethnicity 
(however,  individuals  who  reside  far  from  Ohio  were  very  unlikely  to  participate).  The 
instruments  were  offered  in  English  only,  and  sufficient  comprehension  of  the  English 
language  was  assumed.  Purposive  sampling  as  for  this  study  can  be  very  helpful  when 
the  researcher  needs  to  reach  a  targeted  sample  in  a  reasonable  amount  of  time  and  where 
sampling  for  proportionality  is  not  the  primary  concern  (Jackson,  2009;  Moore,  2007). 
Elsing  purposive  sampling,  the  researcher  is  likely  to  get  the  answers  from  the  target 
population,  but  this  also  increases  the  risk  of  getting  overweight  in  subgroups  in  the 
population  that  are  more  readily  accessible  (Jackson,  2009). 
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In  keeping  with  basie  human  subjeets’  researeh  proteetions,  this  study  relied  on 
volunteers  from  the  population,  with  the  total  number  goal  at  the  minimum  neeessary  to 
aeeomplish  study  goals  based  on  a  priori  power  analysis.  The  first  step  in  the  sample  size 
caleulation  for  this  type  of  study  is  determining  or  estimating  the  effeet  size.  The  effect 
size  is  the  potency  of  the  intervention  or  the  strength  of  the  relationship  under 
investigation  (Cohen,  1992).  By  calculating  an  effect  size  as  part  of  the  analysis  of  the 
data  it  can  assist  in  determining  if  the  research  has  found  something  meaningful  (not 
merely  statistically  significant).  Therefore,  in  advance  of  doing  the  study  one  must 
estimate  the  effect  size  in  the  study  (NCU,  2011).  Lipsey  and  Hurley  (2009)  describe  a 
way  to  estimate  effect  size  by  reviewing  the  literature  on  the  same  or  similar  relationships 
or  interventions  to  find  the  range  of  relevant  effect  sizes  to  estimate  the  effect  size  for 
one’s  study. 

In  the  literature,  usability  problems  or  excelling  attributes  with  varying  display 
designs  (the  varying  of  task  difficulty)  are  likely  to  be  detected  with  an  effect  size  of 
approximately  0.4  (Jacko,  2012;  Lewis,  1994,  2001,  2006;  Nielsen,  Jakob,  &  Landauer, 
1993;  Nielsen,  2000,  2004;  Sauro  &  Lewis,  2005;  Virzi,  1990;  Smith,  2001).  Caulton 
(2001)  indicated  that  a  modest  effect  estimate  such  as  this  should  only  be  used  if  given  a 
firm  homogeneity  assumption  -  that  is  the  types  of  participants  being  sampled  will  have 
the  same  probability  of  encountering  usability  issues  based  on  the  characteristics  of  the 
participants  and  therefore  the  resultants  were  influenced  by  that  modest  effect  estimate. 
The  assumption  of  homogeneity  is  reasonable  given  the  population  of  selected  Ohio 
ANG  pilots  flying  the  same  type  aircraft;  consequently,  the  moderate  effect  size  of  0.4  is 
appropriate  for  power  analysis. 
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By  convention,  studies  in  human  factors  psychological  research  that  are  similar  in 
nature  to  this  study,  i.e.  performed  in  aeronauties  with  aetual  pilots  in  a  simulator  or 
actual  flight,  use  no  more  than  8  to  12  participants  (AFFSA,  2007;  Gallimore  &  Liggett, 
2001;  Liggett  &  Gallimore,  2001,  2002,  2003).  Rather  than  relying  solely  on  this 
conventional  wisdom,  for  this  researeh  G*Power  3.19  was  used  to  eonduet  a  priori  power 
analysis.  The  appropriate  statistieal  test  ehosen  was  the  ANOVA:  Repeated  measures, 
within  faetors.  The  same  pilots  flew  using  the  two  different  display  designs  in  order  to 
vary  task  diffieulty,  and  exhibited  a  range  of  the  three  visual  abilities  (pereeptiveness, 
attentiveness,  and  spatial  memory)  resulting  in  a  2  (displays)  x  3  (visual  abilities)  design, 
and  (2x3)  six  eells  (termed  groups  in  G*Power)  total.  This  study  was  searehing  for  any 
signifieant  effeets  by  these  independent  variables  on  the  dependent  measures  of  situation 
awareness.  The  estimated  Effeet  Size  is  moderate  at  0.40  which  is  the  effeet  size 
measure  indieating  the  total  varianee  explained  by  the  independent  variables  and 
interaetions.  Eaeh  condition  should  be  and  was  repeated  a  minimum  of  four  times. 

A  priori,  eomputed  sample  size  required  for  a  given  power,  alpha,  and  effect  size 
was  ealeulated,  using  the  ANOVA:  Repeated  measures,  within  factors  with  a  moderate 
effeet  size  of  0.40,  the  power  was  set  to  a  minimum  to  the  desired  level  post  hoc  of  0.8 
and  the  alpha  error  probability  was  set  to  0.05  (for  one  tail)  yielding  a  minimum  sample 
size  of  12  participants  and  a  critical  E  of  3. 1599.  This  suggests  that  approximately  12 
participants  should  result  in  an  adequately  powered  study  (see  Table  4  A  Priori 
Computed  Required  Sample  Size). 
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Table  4 

A  Priori  Computed  Required  Sample  Size 


F  tests  -  Factorial  ANOVA;  Repeated  measures,  within  factors 


Analysis:  A  priori:  Computed  Required  Sample  Size 


Input: 

Effect  size  f 

0.4 

a  err  prob 

0.05 

Power  (1-P  err  prob) 

0.8 

Number  of  Groups 

6 

Number  of  Measurements 

4 

Corr  among  rep  measures 

0.75 

Noncentrality  parameter  X 

30.72 

Output: 

Critical  F 

3.16 

Numerator  df 

3.00 

Denominator  df 

18.0 

Total  Sample  Size 

12 

Actual  Power 

0.99 

Table  4  shows  the  results  of  a  priori  power  analysis  from  G*power,  power  =  0.99. 

Demographic  information  was  gathered  for  each  participant:  year  of  birth,  crew 
position,  number  of  years  of  flight  experience,  total  number  of  flight  hours,  gender, 
education,  and  flying  hours  in  aircraft  being  studied.  Participant  age  was  calculated  as 
year  of  birth  subtracted  from  2014. 
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Materials/Instruments 

The  chosen  procedures,  instruments,  and  measurements  are  described  under 
Operational  Definitions  of  Variables.  They  were  minimally  intrusive,  and  the  instruments 
that  were  used  have  been  well-established  in  previous  research  (Harbour  et  ah,  2013). 

The  data  was  collected  in  actual  flight  operations  to  maximize  the  ecological  validity  of 
this  study  (Harbour  et  ah,  2013).  In  order  to  provide  robust  measurements  of  situation 
awareness  and  workload  to  support  the  correlational  analyses,  multiple  subjective  and 
psychophysiological  measures  were  taken  from  each  participant.  This  enabled  a  holistic 
approach  (Hankins  &  Wilson,  1998). 

Participants  were  given  a  computerized  test  for  inherent  visual  attention,  as  the 
Integrated  Visual  and  Auditory  Continuous  Performance  Test  Plus  [IVA-l-];  Brain  Train, 
2010;  Sandford  &  Turner,  1994,  1995)  and  they  were  given  a  test  for  Visuospatial 
Working  Memory  (Spreen  &  Strauss,  2006).  Both  tests  were  presented  on  a  laptop  PC 
with  17”  display  with  an  external  mouse  and  built-in  keyboard  for  input  and  a  headset. 
For  the  IVA-H,  the  validity  of  a  participant’s  scores  was  checked  independently  for 
auditory  and  visual  modalities,  therefore,  utilizing  only  the  visual  modality  portion  was 
valid.  The  IVA-H  is  a  13  to  15  minute  test  that  is  comprised  of  500  trials,  normalized 
from  5-90  years  of  age,  has  demonstrated  concurrent  validity  (>90%)  with  other 
instruments,  is  malingering  proofed,  and  exhibits  no  substantial  practice  or  learning 
effects  (Anatasi  &  Urbina,  1997;  Brain  Train,  2010;  Sanford  &  Turner,  1994,  2001).  For 
the  IVA,  as  similarly  employed  by  commonly  used  IQ  tests,  all  quotient  scores  are 
normalized  and  have  a  mean  score  of  100  and  a  Standard  Deviation  (SD)  of  15.  By  this 
definition,  approximately  68  percent  of  the  general  population  scores  between  85  and 
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115,  which  is  within  one  standard  deviation  of  the  mean.  The  IVA  is  a  ratio  level  of 
measurement  with  a  true  zero  and  a  maximum  of  200.  This  applies  to  both  Va  and  Vp. 

The  Visuospatial  Working  Memory  test  (spatial  n-back)  is  a  two-back  forced 
choiee  computerized  test  paradigm  in  whieh  the  participant  is  presented  with  a  black  X, 
which  can  appear  at  any  of  five  different  locations  on  the  screen  (AFRL,  2009;  Gevins  & 
Cutillo,  1993;  Sohn  &  Doane,  1997,  2000,  2003).  Participants  observe  a  sequence  of 
presentations  of  the  X  at  the  different  locations;  their  task  is  to  compare  the  eurrent 
position  of  the  X  to  where  the  X  appeared  two  presentations  prior.  This  is  done  on  a 
continuous  basis,  requiring  the  partieipants  to  hold  the  two  previous  presentation 
locations  in  spatial  working  memory.  This  test  is  repeated  multiple  times,  over  five 
minutes.  The  spatial  n-back  task  is  a  test  of  the  participant’s  ability  to  retain  spatial 
information  and  to  manipulate  remembered  items  in  working  memory  (AFRL,  2009; 
Sohn,  &  Doane,  1997,  2000,  2003).  It  is  reliable  across  people  and  studies  (Owen, 
McMillan,  Laird,  &  Bullmore,  2005);  see  Table  5. 

The  Ttesting  also  consisted  of  standard  subjective  inventories  delivered  as  peneil 
and  paper  instruments  (Bedford  Workload  Scale,  Roseoe,  &  Ellis,  1990;  China  Lake 
Situation  Awareness  Scale,  Adams,  Kane,  &  Bates,  1998),  and  objeetive 
psyehophysiological  measures  such  electrocardiographic  analysis  of  heart  rate  (HR)  and 
heart  rate  variability  (HRV)  during  flight  (Hankins  &  Wilson,  1998;  Nickel  & 
Nachreiner,  2003),  and  the  eleetroencephalogram  (EEG)  analysis  of  theta  and  alpha 
brainwaves  in-flight  (Wilson,  Estepp,  &  Davis,  2009;  see  Table  5).  The  Bedford 
Workload  seale  is  appropriate  for  in-flight  evaluations,  additionally  it  is  sensitive  and 
reliable  (Handbook  of  Elight  Deek  Researeh,  1995).  This  survey/questionnaire  is  based 
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on  a  binary  decision  tree  structure.  The  participants  are  asked  to  self-assess  whether:  (1) 
it  was  possible  to  complete  the  task,  (2)  the  workload  was  tolerable,  and  or  (3)  the 
workload  was  satisfactory  without  reduction.  The  rating-scale  end  points  are  ‘  1  - 
workload  insignificant’  to  ’  10  -  task  abandoned.  For  this  study,  the  Bedford  definition  of 
workload  focused  on  the  mental  effort  required  to  satisfy  the  perceived  demands  of  a 
specified  task.  Spare  capacity  was  used  to  define  levels  of  workload.  The  Bedford  has 
demonstrated  its  value  in  several  trials  and  for  the  purpose  of  the  research  it  was  apropos 
that  it  have  an  interval  level  of  measurement  (Roscoe  &  Ellis,  1990;  Handbook  of  Flight 
Deck  Research,  1995). 

The  Situation  Awareness  rating  scale  known  as  the  China  Lake  Situational 
Awareness  (CLSA)  can  be  used  for  in-flight  or  post-flight  data  collection  (Roscoe  & 
Ellis,  1990;  Handbook  of  Flight  Deck  Research,  1995).  The  CLSA  is  administered 
similar  to  the  Bedford  Workload  Scale;  it  has  a  unidimensional  rating  scale  and  a  clearly 
defined  set  of  criteria  for  ratings.  The  CLSA  has  high  face  validity,  is  easily  understood 
and  straightforward  to  administer  (Handbook  of  Flight  Deck  Research,  1995)  and  is 
based  on  the  Bedford  WL  scale.  Confidence  has  been  well  established  for  the  validity 
and  reliability  of  the  CLSA,  and  the  scale  has  been  duly  normed  against  the  USAF  pilot 
subject  population  (Roscoe  &  Ellis,  1990;  Handbook  of  Flight  Deck  Research,  1995). 

For  the  purpose  of  the  research  it  was  apropos  that,  the  CLSA  use  an  interval  rating  scale 
with  five  possibilities:  1  being  Very-Good  and  5  being  Very-Poor.  Each  point  on  the 
five-point  scale  refers  to  a  description  of  the  SA  experienced  (CLSA;  Adams,  Kane  & 


Bates,  1998).  See  Table  5. 
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The  Bedford  Workload  Seale  and  China  Lake  Situation  Awareness  Seale  were 
administered  to  the  pilots  post-flight,  in  order  to  avoid  interferenee  with  piloting  tasks. 
During  flight,  heart  rate  was  measured  with  the  use  of  three-lead  ECG  (Hankins  & 
Wilson,  1998;  Nickel  &  Nachreiner,  2003).  The  objective  measures  obtained  from  ECG 
included  heart  rate  (HR)  and  heart  rate  variability  (HRV).  Both  measures  have  been 
extensively  studied.  The  electrocardiographic  data  was  collected  using  a  Vitaport  system 
(Temec  Instruments  B.V.,  Kerkrade,  Netherlands),  which  is  a  small,  portable  pilot-worn 
physiological  data  collection  system  with  onboard  digital  data  storage.  The  three  leads 
were  placed  on  left  and  right  clavicles  and  sternum;  impedances  were  verified  at  or  below 
40  kilo-ohms.  After  completing  written  informed  consent,  participants  were 
instrumented  for  the  ECG.  The  researcher  observed  pilot  activities  onboard  from  the 
jump  seat  and  use  dedicated  marker  channels  along  with  written  notes  to  time  stamp  and 
categorize  flight  segments  and  events  of  interest.  The  HR  and  HRV  data  was  analyzed 
by  extracting  5-minute  segments  during  each  maneuver.  ECG  data  was  band-pass 
filtered  from  .4  Hz  to  30  Hz.  R  wave  peaks  were  marked  using  QRS  tool  based  on 
threshold  detection  followed  up  with  visual  inspection  and  correction.  Inter-beat 
intervals  were  exported  for  subsequent  analyses  of  average  HR  and  SDNN  (in  this  case, 
standard  deviation  of  the  R-R  interval)  by  way  of  Eourier  Transform  analysis  using 
MATLAB. 

Psychophysiology  measurements  for  EEG  and  ECG  have  demonstrated  83.4  to 
85%  accuracy  in  measuring  mental  workload  (Wilson  &  Russell,  2003;  Wilson,  et.  al, 
2009).  The  heart’s  rhythmic  activity  to  include  heart  rate  (HR)  is  influenced  by  higher- 
order  brain  centers  (Wilson,  2002).  Cognitive  activity  will  change  the  rhythmic  patterns 
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of  the  heart,  whieh  is  heart  rate  variability  (HRV);  therefore,  as  cognitive  activity  occurs, 
such  as  mental  workload,  it  causes  a  change  in  HRV  (Wilson,  2002).  Increased  mental 
workload  will  lead  to  an  increase  in  sympathetic  nervous  system  activity,  and  a  decrease 
in  parasympathetic  nervous  system  activity  for  heart  beat  regulation  (Wilson,  2002). 
Consequently,  HR  increases  as  mental  WL  increases  and  HRV  decreases  as  WL 
increases  (Wilson,  2002). 

Electroencephalography  (EEG),  as  a  psychophysiological  measure,  has  been 
successfully  used  to  estimate  an  operator’s  cognitive  state  (Wilson,  2002).  Topographic 
EEG  data  has  shown  changes  in  the  scalp-recorded  patterns  of  alpha  activity  that  are 
consistent  with  the  mental  demands  of  the  various  segments  of  the  tasks  being  performed 
(Wilson,  in  press).  Spontaneous  EEG  has  been  conventionally  classified  into  five 
clinical  frequency  bands,  derived  via  Eourier  transform  of  time-series  EEG  data;  delta 
(0.5-4  Hz),  theta  (4-8  Hz),  alpha  (8-12  Hz),  beta  (12-31  Hz),  and  gamma  (31-43  Hz) 
(Wilson,  2002).  To  determine  a  person’s  cognitive  state  for  use  in  applications  such  as 
workload  assessment  and  adaptive  automation,  the  spontaneous  EEG  has  been  widely 
used,  even  without  specific  events  from  which  event  related  activity  could  be  obtained 
(Wilson,  2002). 

Operational  Definition  of  Variables 

The  independent  variables  for  this  study  were  the  pilot  neurocognitive  factors 
visual  attentiveness  (Va),  visual  perceptiveness  (Vp),  and  visuospatial  working  memory 
(Vswm)  (Task  difficulty  was  manipulated  by  varying  which  display  is  used:  HDD  or  HUD, 
and  was  a  controlled  covariate  variable  but  was  not  a  specific  independent  variable).  The 
dependent  variable  was  both  an  objective  and  subjective  measure  of  situation  awareness. 
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The  three  different  IVs  (Va,  Vp,  Vswm)  were  logieally  eategorized  (eategorieal  level 
variables)  for  the  Faetorial  ANOVA.  They  were  eategorized  as  Low,  Medium,  and  High, 
and  plaeed  into  3  bins  as  a  funetion  of  Z-seore  (Low  <  -1,  -1  <  Medium  <  +1,  +  1  <  High) 
for  three  groups  for  a  total  of  12  subjects  (repeated  measures  within  subjects).  This 
binning  process  renders  the  IV  categorical  and  is  a  relatively  common  way  to  reduce 
nuisance  error  in  the  measurements.  Additionally,  the  Scheffe  Post  Hoc  tests  were  run. 
However,  for  the  (Pearson)  correlation,  and  multiple  regression  modeling,  these  three  IVs 
were  treated  as  a  ratio  level  of  measurement  (ratio  level  variables)  per  the  IV A+.  The 
following  are  operational  explanations  of  the  independent  variables,  dependent  variable, 
and  covariates  used  for  the  study  (see  Table  5). 
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Table  5 


Variable  Operation  and  Level 


Independent  Variables 

Dependent  Variables 

Operation 

Level 

Operation 

Eevel 

Attentiveness  (Va)^ 
Visual  Ability 

Ratio 

(Normalized  0  - 
200) 

China  Lake  (SA) 

Interval: 

1  -  Very 

Good  to 

5  -  Very  Poor 

Perceptiveness  (Vp)  ^ 
Visual  Ability 

Spatial  Working 
Memory  (V^wm)^ 
Visual  Ability 

Ratio 

(Normalized  0  - 
200) 

Ratio 

(0  -  100%  Correct) 

Bedford  (SA)* 

Interval: 

1  -  Workload 

Insignificant 

to 

10  -  Task 
Abandoned, 
Pilot  unable 
to  apply 
sufficient 
effort. 

ECO  (WL)*^ 

Ratio  (0  to 

200): 

Heart  Rate 
(bpm) 

Heart  Rate 
Variability 
(ms) 

EEC  (WL)*^ 

Ratio  (0  to 
200pV): 

Brain  Wave 
Erequency 

Band 

Amplitude 

*Work  Load  can  be  an  implicit  measure  of  Situation  Awareness  (SA)  (Gutzwiller  & 
Clegg,  2012).  Task  manipulation  was  accomplished  by  utilizing  HUD  vs.  HDD,  but  was 


99 


not  counted  as  an  IV.  Task  difficulty  was  varied  by  display  use  manipulation  (a 
eontrolled  eovariate  variable)  so  the  effects  of  visual  abilities  (Va,  Vp,  and  Vswm)  on  the 
level  of  SA  could  be  more  realistically  studied. 

^Composite  SA  seore  between  the  two  surveys  was  verified  by  eorresponding  HR,  HRV, 
and  EEG  objective  data.  Observation  notes  were  taken  of  pilot  performance  as  well. 

X 

The  3  different  IVs  (Va,  Vp,  Vswm)  were  logieally  eategorized  (eategorieal  level 
variables)  for  the  Eactorial  ANOVA  repeated  measures  within  subjeets.  They  were 
binned  into  Eow,  Medium,  and  High,  and  plaeed  into  3  bins  of  subjects  per  group  for  a 
total  of  12  subjeets  (repeated  measure  within  subjeets).  This  binning  proeess  renders  the 
IV  eategorieal  and  is  a  relatively  eommon  way  to  reduee  nuisanee  error  in  the 
measurements.  They  were  eategorized  as  Eow,  Medium,  and  High,  and  plaeed  into  3  bins 
as  a  funetion  of  Z-score  (Eow  <  -1,  -1  <  Medium  <  +1,  +  1  <  High)  for  3  groups  for  a 
total  of  12  subjeets  (repeated  measures  within  subjeets).  Additionally,  the  Seheffe  Post 
Hoe  tests  were  run.  However,  for  eompleteness  the  (Pearson)  eorrelation,  and  linear 
multiple  regression  modeling  these  3  IVs  were  indeed  treated  as  a  Ratio  Eevel  of 
measurement  (ratio  level  variables)  per  the  IV A+. 

Table  5  illustrates  the  independent  and  dependent  variables  along  with  their  operation  and 
level. 


Display  Used  (Controlled  Covariate  Variable).  This  was  the  quality  of  the 
display  design  and  was  varied  to  induee  ehanges  in  task  diffieulty.  There  were  two  levels 
that  were  used  for  this  study:  HUD  and  or  HDD.  In  essenee,  these  levels  vary  the  spatial 
separation  between  a  pilot’s  display  and  his  or  her  normal  out  the  window  gaze  (see 
Eigures  3,  4,  and  5).  The  level  of  measurement  was  nominal  and  the  EARO  arm 
(Harbour  &  Hudson,  2012)  in  3 -dimensional  spaee  measured  spatial  separation  with  the 
units  of  millimeters  and  degrees  in  order  to  sort  this  into  two  eategories.  The  HUD  is 
near  optimal  and  eentered  on  the  pilot’s  EOV  plus  or  minus  5  degrees;  it  was  expeeted  to 
be  psychologieally  non-refraetory  (Harbour,  Hudson,  &  Zehner,  2012).  The  Head  Down 
Display  (HDD)  has  an  ID  design  that  is  suboptimal  and  well  out  of  pilot  line-of-sight  (a 
35  degree  vertieal  drop)  (Harbour,  Hudson,  &  Zehner,  2012).  This  design  was  expeeted 
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to  induce  psychological  refractory  periods  (PRPs,  Proctor  &  Vu,  2010)  due  to  the  need 
to  switch  between  the  HDD  and  the  upper  view  of  the  outside  world.  This  large  drop  was 
expected  to  produce  correspondingly  more  frequent  PRPs.  Consequently,  the  variation  in 
the  spatial  spread  of  information  displays  in  the  cockpit  was  expected  to  induce  variation 
in  task  difficulty,  in  order  to  observe  differential  effects  on  subjective  and 
psychophysiological  measures.  This  task  difficulty  was  varied  by  display  use 
manipulation  (a  controlled  covariate  variable)  so  the  effects  of  visual  abilities  (Va,  Vp, 
and  Vswm)  on  the  level  of  SA  could  be  more  realistically  studied.  This  was  coded  by 
HDD  =  High  Task  Difficulty  =  2,  and  HUD  =  Low  Task  Difficulty  =  1 .  (see  Figures  3,  4, 
and  5,  and  Table  5). 
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ID  Utilized 


In  Persnective 


HUD  1 - 

HUD  2 - > 


Figure  3.  Photograph,  pilot’s  LOS  for  HUD  vs.  HDD.  This  display  manipulation  induced 
variation  in  task  difficulty,  to  observe  differential  effects  on  both  subjective  and  objeetive 
psyehophysiologieal  measures  (Capo-Aponte  et  ah,  2009;  Proctor  &  Vu,  2010). 
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Figure  4.  Photograph,  pilot’s  view  through  the  HUD  and  the  outside  environment.  LOS 
matches  FOV  creating  small  to  zero  PRP,  and  consequently  low  task  difficulty. 
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Figure  5.  Photograph,  pilot’s  view  at  the  HDD  (photo  taken  by  researeher  from  jump 
seat).  LOS  does  not  match  FOV  creating  large  PRP,  and  consequently  high  task 
difficulty. 

Neurocognitive  Variables  (IVs),  There  are  three  Independent  Variables  (IVs): 
Visual  Attention  (visual  attentiveness  and  visual  perceptiveness)  [Va,  Vp]  and 
Visuospatial  Working  Memory  [Vswm]  for  the  primary  constructs  of  situation  awareness 
and  workload  between  the  ID  used.  Visual  Attention  was  measured  by  the  IV A+.  The 
IVA  is  based  on  a  model  that  conceptualized  attention  as  a  multidimensional  capacity 
that  includes  five  primary  factors  including  focused,  sustained,  selective,  alternating  and 
divided  attention.  Visual  attentiveness  and  visual  perceptiveness  are  quotient  scores  on 
the  IV A+  scale,  and  therefore,  the  variables  are  reported  as  a  standard  score  and  have  a 


ratio  level  of  measurement.  These  standard  scores  have  a  mean  of  100  and  a  standard 
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deviation  of  15.  The  test  is  integrated  for  both  auditory  and  visual  performance,  and  it 
was  administered  in  that  manner  in  order  to  preserve  its  validity  and  reliability.  However, 
it  does  yield  separate  visual  and  auditory  scores,  for  this  research  only  the  visual  portion  / 
scores  were  utilized  (see  Table  5). 

Visual  Attentiveness  (IV),  It  is  a  composite  quotient  score  that  is  derived  from 
visual  Vigilance,  Focus,  and  Speed,  involving  both  static  and  dynamic  visual  processing 
(Brain  Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995; 
Sandford  &  Turner,  1994).  Visual  vigilance  is  the  maintenance  of  attention  required  to 
respond  to  a  change  in  the  environment  (a  state  of  inhibition)  such  as  responding  to  a 
target.  Intuitively,  the  mind  can  wander;  the  subject  must  maintain  his  or  her  attention  in 
order  not  to  miss  a  target.  Visual  focus  is  the  consistency  (lack  of  variance)  in  reaction 
time  to  a  change  in  the  environment  and  reflects  the  sustaining  and  maintaining  of 
attention  (Brain  Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine,  &  Goldman, 
1995;  Sandford  &  Turner,  1994).  Visual  speed  is  defined  as  the  average  reaction  time  to 
changes  in  the  environment.  This  measures  discriminatory  mental  processing  speed. 

Both  are  measured  in  milliseconds.  To  elaborate.  Visual  Attentiveness  (Va)  is  the  ability 
to  concentrate  and  be  devoted,  to  be  diligent  and  detailed,  alert,  watchful,  and  responsive. 
This  variable  is  derived  from  visual  Vigilance,  Focus,  and  Speed  scales  (Sandford 
&Tumer,  2009).  For  salience.  Vigilance  is  a  measure  of  inattention  as  evidenced  by  two 
different  types  of  errors  of  omission  (seeks  traits  consisting  of  being  intense  and 
accurately  responsive).  To  be  vigilant,  an  individual  must  maintain  and  direct  attentional 
effort  to  classify  each  stimuli  as  either  a  target  or  foil  and  then  make  the  appropriate 
response  (Sandford  &Turner,  2009).  Focus  is  the  second  major  factor  for  Va  and  reflects 
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the  variance  in  reaction  time  speed  measuring  the  ability  to  sustain  and  maintain 
attention.  The  third  major  factor  is  Speed  reflects  the  average  reaction  time  for  correct 
trials  throughout  the  test,  thus  Speed  measures  discriminatory  mental  processing  speed. 
Speed  equals  =  Mean  visual  reaction  time  in  milliseconds  for  all  correct  trials  (Sandford 
&Tumer,  2009).  The  Attentiveness  score  is  very  apropos  to  assessing  inherent  visual 
attention  of  the  pilot  because  these  factors  are  essential  characteristics  of  a  pilot  or  UAV 
operator  when  processing  information  from  a  display. 

Visual  Perceptiveness  (IV),  Visual  perceptiveness  is  similarly  a  composite  score 
based  on  three  sub-scores.  It  consists  of  visual  Prudence,  Consistency,  and  Stamina  and 
involves  both  static  and  dynamic  visual  processing  (Brain  Train,  2010;  Brain  Train,  2010; 
Corbett  &  Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner, 
1994).  Visual  prudence  is  the  selection  or  choice  of  a  correct  response  or  responses  in 
the  environment  (non-inhibited).  That  is  being  non-impulsive,  the  ability  to  not 
automatically  react  yielding  an  incorrect  response  in  the  changing  environment  (Brain 
Train,  2010;  Corbett  &  Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995;  Sandford 
&  Turner,  1994).  Visual  consistency  is  the  ability  to  stay  on  task,  respond  reliably, 
making  dependable  responses  in  a  dynamic  environment  (Brain  Train,  2010;  Corbett  & 
Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner,  1994).  Visual 
stamina  is  the  lack  of  variability  in  a  subject’s  response  times  in  the  environment.  This  is 
the  ability  to  maintaining  a  sustained  effort  over  time.  (Brain  Train,  2010;  Corbett  & 
Constantine,  2007;  Sandford,  Fine,  &  Goldman,  1995;  Sandford  &  Turner,  1994).  These 
are  measured  in  milliseconds.  To  elaborate.  Visual  Perceptiveness  (Vp)  is  the  ability  to 
have  a  perceptivity,  to  be  insightful,  and  to  have  discernment  -  the  ability  to  perceive  that 
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which  is  obscured.  For  salience,  Visual  Perceptiveness  is  visual  insightfulness  or 
perceptivity  and  it  is  equivalent  to;  Prudence  as  a  measure  of  Carefulness  (anti- 
impulsivity)  and  response  inhibition  as  evidenced  by  three  different  types  of  errors  of 
commission(seeks  traits  of  consisting  of  being  circumspective  and  mindful),  Consistency, 
which  measures  the  general  reliability  and  variability  of  response  times  and  is  used  to 
help  measure  the  ability  to  stay  on  task,  and  Stamina,  which  compares  the  mean  reaction 
times  of  correct  responses  during  the  first  200  trials  to  the  last  200  trials.  This  score  is 
used  to  identify  levels  of  sustaining  attention  and  effort  over  time  (Sandford  &  Turner, 
1994,  1995).  The  Perceptiveness  score  is  very  apropos  to  assessing  inherent  visual 
attention  of  the  pilot  because  these  factors  are  essential  characteristics  of  a  pilot  or  UAV 
operator  when  processing  information  from  a  display. 

Visuospatial  Working  Memory  (IV),  This  is  working  memory  that  contains 
visual  and  spatial  information  that  is  stored  in  the  visuospatial  sketchpad  in  the  mind 
(Baddeley,  2006),  this  immediate  memory  can  be  thought  of  as  a  workbench  where 
material  is  continuously  being  combined  and  transformed.  Visuospatial  working  memory 
was  assessed  via  the  spatial  n-back  task  (Gevins  &  Cutillo,  1993),  already  explained  in 
detail  in  this  section  along  with  its  validity.  This  variable’s  level  of  measurement  is  ratio 
and  is  a  percent  of  selections  that  are  correct  out  of  the  total  number  of  trials.  Scores  can 
range  from  0%  correct  to  100%  correct. 

Situation  Awareness  (DV),  This  study  was  focused  on  situation  awareness,  and 
it  is  the  psychological  ability  and  capacity  to  perceive  information  and  act  on  it 
acceptably.  Previous  research  (Gutzwiller  &  Clegg,  2012;  Svensson  &  Wilson,  2002) 
had  also  shown  that  mental  workload  is  a  component  and  predictor  of  situation 
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awareness;  eonsequently,  workload  was  assessed  as  well  to  provide  additional  implicit 
measurements  of  SA  for  the  ANOVA  test.  Mental  workload  is  an  implicit  measurement 
of  situation  awareness;  as  mental  workload  increases,  cognitive  situation  awareness 
decreases  (Gutzwiller  &  Clegg,  2012;  Kokar  &  Endsley,  2012;  Svensson  &  Wilson, 

2002;  Wilson  &  Russell,  2003;  Wilson,  et.  al,  2009).  The  Bedford  Workload  Scale  and 
China  Lake  Situation  Awareness  Scale  were  administered  to  the  pilots.  This  was  the 
Dependent  Variable  (DV)  and  were  the  results  from  the  self-reported  levels  of  situation 
awareness  and  workload  (Cohen  &  Cohen,  1983;  Cohen,  1960),  and  as  validation  and 
verification  Heart  Rate  Variability  (HRV),  Heart  Rate  (HR),  the  EEC  (alpha  and  theta 
wave  divergence)  were  measured  as  well. 

Electroencephalography  (LEG),  as  a  psychophysiological  measure,  has  been 
successfully  used  to  estimate  an  operator’s  cognitive  state  (Wilson,  2007).  Eirst  studied 
in  animals  by  Richard  Catton  in  1875,  electroencephalography  is  the  study  of  the 
electrical  activity  of  the  brain  (Wilson,  2007).  It  has  been  determined  that  superposition 
of  post-synaptic  potentials  due  to  volume  conduction  causes  the  observable  electric 
activity  at  the  scalp  (Wilson,  2007).  Topographic  LEG  data  will  show  changes  in  the 
scalp-recorded  patterns  of  alpha  activity  that  are  consistent  with  the  mental  demands  of 
the  various  segments  of  the  tasks  being  performed  (Wilson,  in  press).  There  are  two  ways 
to  study  EEG;  time-locked  to  the  stimulus  (event-related  or  evoked  potential)  or  spectral 
(windowed/averaged  in  time)  (J.C  Christensen,  personal  communication,  Jan  10,  2012) 
(see  Table  5). 

Without  an  external  stimulus,  spontaneous  EEG  occurs,  such  as  alpha  and  beta 
rhythms,  whereas  ERPs  occur  in  response  to  a  specific  stimulus.  Spontaneous  EEG  has 
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been  eonventionally  elassified  into  five  elinieal  frequeney  bands,  derived  via  Fourier 
transform  of  time-series  EEG  data:  delta  (0.5-4  Hz),  theta  (4-8  Hz),  alpha  (8-12  Hz),  beta 
(12-31  Hz),  and  gamma  (31-43  Hz)  (Wilson,  2007).  To  determine  a  person’s  eognitive 
state  for  use  in  applieations  such  as  workload  assessment  and  adaptive  automation,  the 
spontaneous  EEG  has  been  widely  used,  even  without  specific  events  from  which  ERPs 
are  obtained  (Wilson,  2007). 

During  mentally  demanding  portions  of  tasks  frontal  theta-band  EEG  has  shown 
increased  activity  (Wilson,  2007),  therefore,  theta-band  EEG  should  increase  as  mental 
workload  increases  (Wilson,  2007).  Topographic  EEG  data  from  29  channels  illustrated 
decreases  in  parietal  alpha-band  activity,  which  correlated  to  higher  mental  demands 
therefore,  parietal  alpha-band  EEG  should  decrease  as  mental  workload  increases 
(Wilson,  2002).  Multiple  EEG  recording  sites  are  useful  to  detect  significant  changes  in 
regional  brain  activity  that  are  related  to  different  tasks.  It  has  been  found  that  increases 
in  cognitive  workload  were  associated  with  decreased  EEG  alpha-band  power  over 
parietal  scalp  sites  (Wilson,  2002),  with  corresponding  increases  in  theta  power. 
Therefore,  theta  and  alpha  diverge,  when  mental  workload  significantly  increases 
(Hankins  &  Wilson,  1998).  To  summarize,  psychophysiology  measurements  including 
the  combination  of  EGG  and  EEG  have  shown  a  high  level  of  accuracy  and  validity  in 
explicitly  measuring  workload  which  in  turn  is  implicitly  measuring  situation  awareness 
(Gutzwiller  &  Clegg,  2012;  Kokar  &  Endsley,  2012;  Svensson  &  Wilson,  2002;  Wilson 
&  Russell,  2003;  Wilson,  et.  al,  2009).  As  mental  workload  increases,  cognitive  situation 
awareness  decreases  (Gutzwiller  &  Clegg,  2012;  Kokar  &  Endsley,  2012),  consequently, 
as  one  measures  the  current  psychological  and  physiological  state  of  an  individual,  one  is 
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measuring  mental  workload  and  in  turn,  cognitive  situation  awareness  (Gutzwiller  & 
Clegg,  2012). 

Data  Collection,  Processing,  and  Analysis 

The  fundamental  analysis  that  addressed  the  research  questions  was  the  within- 
subjects  and  factors  repeated-measures  factorial  ANOVA  which  was  used  to  measure  the 
main  effects  of  the  independent  variables  (task  difficulty  and  visual  ability)  the 
interactions  among  the  independent  variables,  the  importance  of  the  dependent  variable 
(situation  awareness),  and  the  strength  of  association  between  these  variables.  There  was 
a  check  at  mid-way  to  look  for  dropouts. 

Normality  in  the  data  was  verified  and  a  standard  parametric  statistical  analysis 
was  performed  (Creswell,  2009;  de  Vaus,  2001;  Moore,  2007).  The  ANOVA  was  used  to 
look  for  a  significant  difference  between  means  (Creswell,  2009;  de  Vaus,  2001;  Moore, 
2007).  That  is,  it  was  used  to  see  whether  or  not  there  was  a  significant  difference 
between  the  average  scores  of  the  groups  being  tested. 

In  multivariate  analysis,  the  composite  variable  is  ubiquitous.  The  composite 
variable  is  derived  by  combining  two  or  more  variables  linearly  as  in  path  analysis,  or 
curvilinearly  as  in  polynomial  regression  analysis.  Each  input  variable  is  incorporated  by 
assigning  a  weight  and  or  order  as  appropriate,  utilizing  the  multivariate  technique.  This 
research  derived  the  composite  variable  [Predicted  Situation  Awareness  (cognitive 
situation  awareness  and  mental  workload)],  and  on  the  basis  of  analysis,  this  composite 
variable  was  a  function  of:  Information  Display  (ID)  used.  Visual  Attentiveness,  Visual 
Perceptiveness,  and  Visual  Spatial  Working  Memory  (Grim  &  Yamold,  2006). 
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Assumptions 

The  participants  were  recruited  at  an  Air  National  Guard  base  in  central  Ohio  and 
were  qualified  pilots  in  the  aircraft  being  used  for  the  study.  This  resulted  in  relative 
homogeneity  of  the  population  that  reduced  nuisance  variance.  The  analysis  assumed 
that  there  was  at  least  an  approximate  relationship  between  the  predictors  and  outcome 
variables,  that  participants  were  independent  of  each  other,  and  that  the  scores  were 
approximately  normally  distributed.  In  the  absence  of  prior  research  regarding  the 
relationship  between  visual  abilities,  situation  awareness,  and  cognitive  workload,  a 
relationship  is  a  reasonable  and  pragmatic  assumption.  Participants  were  measured  and 
sampled  independently,  so  it  was  reasonable  to  assume  that  the  scores  were  independent 
of  each  other  and  the  data  were  examined  and  the  data  were  verified  to  be  normally 
distributed. 

Limitations 

Some  of  the  pre-existing  factors  such  as  the  subject  pilot’s  previous  experience 
using  HUDs  and  his  or  her  level  of  piloting  skills  were  not  taken  into  account.  In 
addition,  there  are  an  untold  number  of  possible  outside  influences  that  could  have 
affected  the  results,  such  as  sleep  quality,  family/emotional  disturbance,  diet,  etc. 
(Cervone  &  Pervin,  2007).  Additionally,  there  was  not  complete  randomness.  Even 
with  these  disadvantages,  because  the  shortcomings  were  recognized  throughout  this 
study,  and  repeated  measures  were  taken,  the  results  are  still  valuable  (Creswell,  2009;  de 
Vaus,  2001;  Trochim  &  Donnelly,  2008). 
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Delimitations 

The  primary  purpose  of  this  quasi-experimental  quantitative  study  was  to  test 
visual  pereeptiveness,  attentiveness,  and  spatial  working  memory  as  predictors  of 
situation  awareness.  In  order  to  maximize  ecological  validity  and  minimize  nuisance 
variance  due  to  heterogeneous  participants,  this  project  focused  on  the  controlled 
assessment  of  the  visuocognitive  abilities  of  Air  National  Guard  pilots,  with  actual 
piloting  as  a  completely  real  task  suitable  for  measuring  resulting  situation  awareness. 

This  was  the  most  effective  and  realistic  method  to  perform  the  research  that 
addressed  the  problem  and  the  purpose.  As  an  example  alternative  approach,  a 
completely  laboratory  based  study  utilizing  the  typical  convenience  sample  of  college 
undergraduates  could  have  been  accomplished;  however,  one  would  expect  large 
individual  differences  as  well  as  serious  concerns  about  the  measurement  of  situation 
awareness  outside  any  meaningful  task  context.  The  choice  of  participant  population 
potentially  limits  the  generalization  of  results  to  other  populations,  however  this  was  a 
realistic  and  necessary  delimitation  in  order  to  increase  the  chances  of  observing 
meaningful  results  and  keep  the  study  feasible  in  terms  of  sample  size  and  associated 
costs.  Another  delimitation  was  the  measures  chosen;  while  other  techniques  for 
assessing  SA  could  have  been  used  (e.g.,  SAGAT;  Endsley,  1995)  the  methods  chosen 
were  the  most  practical  and  effective  in  the  aviation  environment.  Longer  surveys  or 
interruptions  were  not  feasible  due  to  real-world  demands,  but  the  surveys  that  were  used 
were  acceptable  given  the  increased  ecological  validity. 
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Ethical  Assurances 

The  ethics  of  this  study  were  covered,  addressed,  and  monitored  throughout  the 
study.  The  subjects  flew  aircraft  with  an  associated  minimal  and  low-risk  level. 

Extensive  consideration  was  given  to  the  possible  risks  involved  and  possible  mitigations 
for  those  risks.  It  was  addressed  as  to  why  the  study  should  be  done  in  the  manner 
chosen  to  conduct  it — ^with  pilots  exposed  to  real  risks,  rather  than  using  simulators.  This 
study  was  conducted  with  an  AFRL  IRB  and  NCU  IRB  approval,  participants  completed 
comprehensive  written  informed  consent,  and  multiple  USAF  board  reviews  were 
performed.  Specifically,  before  the  research  and  data  collection  was  initiated  an  USAF 
Safety  Review  Board  (SRB)  and  an  USAF  Flight  Test  Board  (FTB)  was  conducted  in 
addition  to  two  Institutional  Review  Boards  (IRBs):  one  carried  out  by  the  USAF  and  one 
conducted  by  NCU. 

The  rights,  responsibilities,  and  safety  of  the  researcher  and  subjects  are  and  must 
be  strictly  respected  at  all  times  and  were  for  this  study  (Creswell,  2009).  To  do  this, 
there  were  key  areas  that  were  followed  and  adhered  to  during  and  after  this  research  as 
they  pertain  to  the  ethical  practice  and  relationship  between  researcher,  psychologist,  or 
specialist  and  subject,  or  participant.  Before  psychological  research  can  begin,  informed 
consent  must  be  obtained.  This  was  accomplished  using  an  Informed  Consent  Document 
(ICD)  that  was  read  and  completed  by  each  and  every  subject  involved  in  this  experiment 
(Creswell,  2009).  The  researcher  has  the  responsibility  of  guarding  the  confidentially  of 
each  individual  subject,  and  this  occurred  during  and  after  this  study  (Creswell,  2009). 
The  IVA  and  Vswm  data  obtained  was  for  research  purposes  only  and  was  not  used  in  any 
type  of  medical  capacity,  nor  was  the  ECG  and  EEC  collected  data.  To  name  a  few 
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items,  the  researeher  must  and  did  provide,  participants  information  regarding  the 
research  purpose,  the  procedures,  the  possible  benefits  and  risks  of  adverse  outcomes, 
rights  and  responsibilities  of  subjects,  limits  of  confidentiality  and  what  can  be  expected 
from  the  researcher  (Creswell,  2009). 

Summary 

The  understanding  of  SA  lacked  a  level  of  detail  to  account  for  psychological 
mechanisms  needed  to  predict  the  outcome  (Endsley,  1999;  Jodlowski,  2008).  Even 
though  SA  is  considered  a  cognitive  engineering  construct  there  were  no  direct  objective 
psychophysiological  or  psychological  measurements  for  predictors  of  it.  There  was  no 
consensus  on  predictors  for  SA  and  this  area  had  tended  to  be  elusive. 

This  study  was  intended  to  discover  the  relationships  between  perceptual  effects 
caused  by  ID  designs’  spatial  location,  inherent  neurocognitive  factors,  and  changes  in 
SA  and  WE.  Most  SA  and  WE  studies  had  been  performed  with  the  use  of  the  flight 
simulator  or  in  the  laboratory,  and  used  solely  subjective  measurements.  However,  very 
little  prior  research  had  examined  cognitive  phenomena  (mental  WE  and  resulting  SA),  in 
an  actual  aircraft  during  flight.  Likewise,  previous  research  had  not  examined  the 
relationships  between  individual  visual  abilities  and  resulting  levels  of  SA  and  WE 
during  complex  multitasking. 

A  within-subjects  quasi-experimental  repeated-measure  design  was  used  that 
incorporated  neurocognitive  SA  and  WE  measurement.  All  participants  were  pilots 
qualified  in  the  aircraft.  Subjects  filled  out  a  SA  scale,  a  WE  scale,  and  a  modified 
Primary  Elight  Display  (PED)  evaluation  questionnaire  after  each  sortie  and  the 
psychophysiological  measurements  including  electrocardiography  (ECG)  and 
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Electroencephalography  (EEG)  were  collected  per  sortie  for  comparisons  within  group 
and  between  groups  (HUD  and  HDD).  Results  were  analyzed  using  traditional 
parametric  approaches  (ANOVA,  correlation)  to  test  for  significant  relationships  between 
visual  abilities,  WE,  and  SA,  whether  assessed  subjectively  or  neurocognitively. 
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Chapter  4:  Findings 

Visual  processing  can  be  eategorized  as  static  and  dynamic  (Proctor  &  Vu,  2010), 
and  may  be  operationalized  as  visual  attentiveness  and  perceptiveness,  integrated  with 
visuospatial  memory  (Brain  Train,  2010;  Christensen  et  ah,  2013;  Corbett  &  Constantine, 
2007;  Endsley,  2012;  Gugerty,  in  press;  Sandford,  Fine,  &  Goldman,  1995;  Sandford  & 
Turner,  1994;  Vidulich  &  Tsang,  2012;  Wiekens  &  McCarley,  2008).  Consequently,  the 
primary  purpose  of  this  quasi-experimental  quantitative  study  was  to  test  the  predictive 
value  of  these  specific  candidate  visuo-cognitive  abilities:  (a)  Visual  Attentiveness, 
consisting  of  Vigilance,  Focus,  and  Speed,  (b)  Visual  Perceptiveness,  consisting  of 
Prudence,  Consisteney,  and  Stamina,  and  (c)  Visuospatial  Working  Memory,  which 
consists  of  Working  Memory  that  is  stored  in  the  visuospatial  sketchpad  in  the  mind,  as 
predictors  of  situation  awareness,  e.g.  the  effects  of  visual  ability  on  the  level  of  situation 
awareness.  Therefore,  this  projeet  tested  the  predictive  value  of  these  variables  as 
factors  between  a  particular  task  to  be  performed  and  the  eventual  outcome  of  situation 
awareness,  in  order  to  examine  if  there  was  a  potential  relationship  between  the 
neurocognitive  factors  of  visual  attentiveness,  (Hypotheses  1),  visual  perceptiveness 
(Hypotheses  2),  and/or  visual  spatial  working  memory  (Hypotheses  3)  and  situation 
awareness.  These  neuroeognitive  factors  were  derived  from  visual  proeessing  that  can  be 
categorized  as  static  and  dynamic,  and  may  be  operationalized  as  visual  attentiveness  and 
perceptiveness,  integrated  with  visuospatial  memory  and  measured  by  the  IV A+  and  N- 
back  tests. 

As  Pavlov,  Watson,  and  Skinner  performed  field  experiments  in  order  to 
contribute  to  psychological  theory  (Cervone  &  Pervin,  2007),  a  field  experiment  in  the 
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paradigm  of  neuroergonomics  (Parasuraman,  Christensen,  &  Grafton,  2012)  employing 
quasi-experimental  repeated-measures  (within-participants),  was  used  for  this  study. 

This  further  inquiry  involving  visual  cues  clarified  the  role  of  these  abilities  in  order  to 
make  theory-driven  predictions  for  situation  awareness  (Harbour  et  ah,  2012;  Vidulich  & 
Tsang,  2012;  Wickens  &  McCarley,  2008). 

These  pilots  were  recruited  at  an  airport  in  north  central  Ohio  by  utilizing  a 
recruitment  memo  and  posters  in  squadron  buildings.  All  participants  were  pilots 
qualified  in  the  aircraft.  There  were  two  display  conditions  that  varied  the  difficulty  of 
maintaining  SA  by  changing  the  presence  and  position  of  the  information  display.  The 
first  condition  had  an  Information  Display  (ID)  design  that  was  near  optimal  and  centered 
on  the  pilot’s  field  of  view  and  was  no  more  than  five  degrees  above  or  below  the  pilot’s 
line  of  sight  (Harbour,  Hudson,  &  Zehner,  2012).  The  second  condition  had  an  ID  design 
that  was  suboptimal  and  well  out  of  pilot  line-of-sight  (a  3  5 -degree  vertical  drop) 
(Harbour  et  ah,  2012). 

The  experimental  study  of  situation  awareness  presents  particular  challenges. 
Situation  awareness  is  highly  context-specific;  experimental  testing  and  manipulation 
must  be  conducted  with  respect  to  an  actual  task.  As  previously  discussed,  it  is  a 
complicated  tangible  that  is  not  well  addressed  by  impoverished  laboratory  study 
focusing  on  simple,  artificial  tasks.  Consequently,  the  use  of  a  meaningful,  consequential 
task  context  is  essential  to  the  study  of  situation  awareness.  This  chapter  will  review  the 
findings  of  this  study  by  binding  the  purpose  around  the  research  questions  and  resulting 
hypothesis. 
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This  quantitative  research  utilized  a  quasi-experimental  design  that  made  use  of 
repeated  measures  comparisons,  with  situation  awareness  as  the  dependent  variable.  The 
within-subjects  and  factors  repeated-measures  AN OVA  was  used  to  measure  the  effects 
of  the  variables  (independent  /  dependent),  their  interactions  and  importance,  and  the 
strength  of  association  between  variables.  Each  subject  in  the  experiment  was  exposed  to 
all  levels  and  conditions.  As  mental  workload  goes  up,  cognitive  situation  awareness 
goes  down  (Gutzwiller  &  Clegg,  2012;  Kokar  &  Endsley,  2012),  consequently,  as  one 
measures  the  current  psychological  and  physiological  state  of  an  individual,  one  is 
measuring  mental  workload  and  in  turn,  cognitive  situation  awareness  (Gutzwiller  & 
Clegg,  2012).  Succinctly  put,  as  one  measures  mental  workload  one  implicitly  measures 
situation  awareness  (Gutzwiller  &  Clegg,  2012;  Kokar  &  Endsley,  2012).  In  order  to 
explore  visual  abilities  as  predictors  of  situation  awareness  with  factors  between  task 
difficulty  and  situation  awareness,  the  following  research  questions  sought  to  address  the 
overarching  question  of  identifying  predictors  of  situation  awareness  under  varying  task 
difficulty. 

In  all  three  below  questions,  the  visual  abilities  were  evaluated  as  predictors 
(factors)  of  situation  awareness.  Task  difficulty,  in  and  of  itself,  was  analyzed  as  a 
necessary  manipulation  check  to  verify  that  the  conditions  produced  significant  main 
effects  in  situation  awareness;  however  this  was  not  a  central  research  question.  A  main 
effect  of  visual  ability  was  probed  to  determine  which  abilities  contributed  to  observe 
variation  in  situation  awareness,  likewise  any  interaction  between  visual  abilities  and  task 
difficulty  were  assessed.  Additionally  multiple  regression  analysis  was  performed.  The 
bottom  line  is  what  were  the  effects  of  visual  ability  on  the  level  of  situation  awareness? 
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To  review,  a  detailed  investigation  that  foeused  on  the  following  main  researeh  questions 
was  performed: 

Ql.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  attentiveness  on  situation  awareness? 

Q2.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  perceptiveness  on  situation  awareness? 

Q3.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of 
subject  inherent  visual  spatial  working  memory  on  situation  awareness? 

To  address  the  research  questions,  the  following  hypotheses  were  tested: 

Hlo.  Under  varying  task  difficulty,  there  will  be  no  statistically  significant  effects 
of  inherent  visual  attentiveness,  (as  measured  by  the  Integrated  Visual  and 
Auditory  Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on 
resulting  situation  awareness. 

Hla.  Under  varying  task  difficulty,  there  will  be  statistically  significant  effects  of 
inherent  visual  attentiveness,  (as  measured  by  the  Integrated  Visual  and  Auditory 
Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on  resulting 
situation  awareness. 

H2o.  Under  varying  task  difficulty,  there  will  be  no  statistically  significant  effects 
of  inherent  visual  perceptiveness,  (as  measured  by  the  Integrated  Visual  and 
Auditory  Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on 
resulting  situation  awareness. 

H2a.  Under  varying  task  difficulty,  there  will  be  statistically  significant  effects  of 
inherent  visual  perceptiveness,  (as  measured  by  the  Integrated  Visual  and 
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Auditory  Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on 
resulting  situation  awareness. 

H3o.  Under  varying  task  difficulty,  there  will  be  no  statistically  significant  effects 
of  Visuospatial  Working  Memory  Test  results  (AFRL,  2009),  on  resulting 
situation  awareness. 

H3a.  Under  varying  task  difficulty,  there  will  be  statistically  significant  effects  of 
Visuospatial  Working  Memory  Test  results  (AFRL,  2009),  on  resulting  situation 
awareness. 

A  sample  of  19  C-27J  pilots,  at  the  179*  Airlift  Wing  (AW)  (see  Appendix  A  for 
List  of  Acronyms),  were  tested  with  a  neurological  assessment  to  examine  if 
neurocognitive  factors  could  predict  levels  of  situational  awareness  (see  Appendix  B). 
This  chapter  includes  the  statistical  assumptions  that  were  utilized,  the  descriptive  results 
of  participant  demographic  data,  and  the  procedures  used  to  test  the  three  hypotheses  that 
were  examined  to  describe  the  results  of  the  study. 

The  within-subjects  and  factors  repeated-measures  ANOVA  was  used  to  measure 
the  effects  of  the  variables  (independent  /  dependent),  their  interactions  and  importance, 
and  the  strength  of  association  between  variables.  The  Pearson’s  product-moment 
correlation  was  used  to  examine  the  correlations  and  relationships  between  visual 
abilities  and  situation  awareness.  Multiple  regression  analyses  was  used  to  investigate 
the  predictor  significance  and  variances  of  the  predictors  (i.e.,  independent  variables)  and 
the  dependent  criterion  variable,  in  order  to  derive  a  mathematical  equation  for  SA  as  a 
function  of  Va,  Vp,  and  Vswm-  This  chapter  ends  with  a  discussion,  evaluation,  and 
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synopsis  of  the  findings  regarding  whether  speeifie  neurocognitive  abilities  have  a  role  in 
predieting  and  determining  situational  awareness. 

Assumptions  Made  in  the  Statistical  Analyses 

There  were  several  assumptions  that  underlay  the  statistical  analyses,  before 
project  execution  a  priori  power  analysis  was  performed  with  required  assumptions. 

After  project  implementation,  a  post  hoc  power  analysis  was  performed  and  the  ANOVA 
assumptions  of  normality,  independence,  and  homoscedasticity  were  verified,  along  with 
the  assumptions  of  multiple  regression  analysis  that  include  those  same  assumptions  plus 
linearity,  multicollinearity,  singularity,  absence  of  outliers,  and  statistical  independence 
of  the  errors. 

In  keeping  with  fundamental  human  subjects’  research  protections,  this  study 
relied  on  volunteers  from  the  population,  with  the  total  number  goal  at  the  minimum 
necessary  to  accomplish  study  goals  based  on  a  priori  power  analysis.  Therefore,  in 
advance  of  doing  this  study  one  must  estimate  the  effect  size  in  the  study  (NCU,  2011). 
Lipsey  and  Hurley  (2009)  describe  a  way  to  estimate  effect  size  by  reviewing  the 
literature  on  the  same  or  similar  relationships  or  interventions  to  find  the  range  of 
relevant  effect  sizes  to  estimate  the  effect  size  for  one’s  study.  In  the  literature  usability 
problems  or  excelling  attributes  with  varying  display  designs  (the  varying  of  task 
difficulty)  are  likely  to  be  detected  with  an  effect  size  of  approximately  0.4  (Jacko,  2012; 
Lewis,  1994,  2001,  2006;  Nielsen,  Jakob,  &  Landauer,  1993;  Nielsen,  2000,  2004;  Sauro 
&  Lewis,  2005;  Virzi,  1990;  Smith,  2001).  Caulton  (2001)  indicated  that  a  modest  effect 
estimate  such  as  this  should  only  be  used  if  given  a  firm  homogeneity  assumption  -  that 
is  the  types  of  participants  being  sampled  will  have  the  same  probability  of  encountering 
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usability  issues  based  on  the  characteristics  of  the  participants  and  therefore  the  resultants 
are  influenced  by  that  modest  effect  estimate.  The  assumption  of  homogeneity  is 
reasonable  given  the  population  of  selected  Ohio  ANG  pilots  flying  the  same  type 
aircraft;  consequently,  the  moderate  effect  size  of  0.4  was  appropriate  for  power  analysis. 

G*Power  3.19  was  used  to  conduct  a  priori  power  analysis.  The  appropriate 
statistical  test  chosen  was  the  ANOVA:  Repeated  measures,  within  factors.  The  same 
pilots  flew  using  the  two  different  display  designs  in  order  to  vary  task  difficulty,  and 
exhibited  a  range  of  the  three  visual  abilities  (perceptiveness,  attentiveness,  and  spatial 
memory).  Each  condition  was  repeated  a  minimum  of  four  times. 

Therefore,  a  priori,  computed  sample  size  required  for  a  given  power,  alpha,  and 
effect  size  was  calculated,  using  the  ANOVA:  Repeated  measures,  within  factors  with  a 
moderate  effect  size  of  0.40,  the  power  was  set  to  a  minimum  to  the  desired  level  post 
hoc  of  0.8  and  the  alpha  error  probability  was  set  to  0.05  (for  one  tail)  yielding  a 
minimum  sample  size  of  12  participants  and  a  critical  F  of  3. 1599.  This  suggested  that 
approximately  12  participants  should  result  in  an  adequately  powered  study  (see  Table  4: 
A  priori:  Computed  Required  Sample  Size).  Even  though  there  were  a  total  of  19 
participants  only  12  were  in  each  condition  repeated  four  times;  therefore,  the  statistical 
tests  for  the  hypotheses  and  study  results  answering  the  research  questions  are  based  on 
those  twelve  subjects.  Consequently,  this  led  to  a  decrease  in  the  potential  for  a  Type  1 
error,  in  addition,  this  computed  sample  size  was  associated  with  an  increase  in  the  power 
and  a  reduction  in  Type  11  error  (Cozby,  2006).  The  researcher  did  achieve  statistical 
power  for  statistical  analyses  at  a  power  1-  p  =  0.98,  a  =  .05,  and  effect  size  (see  Table  7 
and  Table  9),  minimizing  Type  1  and  Type  11  errors. 
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Post  hoc,  as  the  dependent  variable  is  eontinuous,  an  effeet  size  of  the  d  family  of 
standardized  mean  differenees  eould  be  eonsidered  (Cohen,  1988).  However,  the  effeet 
size  measure  Cohen’s /  (Cohen,  1988)  is  appropriate  for  ealeulating  the  effeet  size 
within  a  multiple  regression  model  in  whieh  the  independent  variables  of  interest  are 
multivariate  (sueh  as  Va,  Vp,  and  Vswm)  and  the  dependent  variable  (SA)  is  eontinuous 
and  repeated  measures  of  subjeets  utilized  (such  as  4-times)  (Friedmann  et  ah,  2008; 
Selya,  Rose,  Dierker,  Hedeker,  &  Mermelstein,  2012)  .  Cohen’s /  is  eommonly 
presented  in  a  form  appropriate  for  global  effeet  size: 

f  ^  =  - 

^  1-  R2 


Consequently,  the  effeet  size 


r  = 


(0.75)2 
1  -  (0.75)2 


Hence,  therefore 


/2  =  3.05 


Table  6 

Model  Summary 


Model 

Summary 

Change  Statistics 

R 

Adjusted 

Std.  Error 
of  the 

R 

Square  F 

Sig.F 

R  Square 

R  Square 

Estimate 

Change 

Change  dfl  df2 

Change 

.868"  .753 

.660 

.18356 

.753 

8.125  3  8 

.008 

a.  Predictors:  (Constant),  Vswm,  Va,  Vp 


Table  6  illustrates  that  together  Vswm,  Va,  Vp,  account  for  75.3  percent  of  the  varianee  in 
SA,  p<.05. 
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Table  7 

Post  hoc:  Computed  Achieved  Power 


F  tests  -  Linear  multiple  regression:  Fixed  model,  inerease 


Analysis:  Post  hoe:  Compute  aetiieved  power 


Input: 

Effect  sizey^ 

3.05 

a  err  prob 

0.05 

Total  sample  size 

12 

Number  of  tested  predictors 

3 

Total  number  of  predictors 

3 

Output: 

Noncentrality  parameter  k 

36.6000000 

Critical  F 

4.0661806 

Numerator  df 

3 

Denominator  df 

8 

Power  (1-P  err  prob) 

0.9831082 

Table  7  illustrates  the  Post  hoe:  Computed  Aehieved  Power  as  eomputed  by  G*power, 
Power  =  0.9831. 

The  goal  of  hypothesis  testing  with  the  ANOVA  is  to  determine  whether  the  means 
of  the  sample  differ  more  than  you  would  expect  if  the  null  hypothesis  were  true.  The 
tests  were  statistically  significant  for  the  three  predictor  visual  abilities  Vp,  Va,  and  Vswm 


as  task  difficulty  was  varied,  F(3,l  1)  =  8.125, />  =  .008.  Specifically,  Va  [F(2,l  1)  = 
5.749,;?  =  .025],  Vp  [F(2,l  1)  =  12.125,;?  =  .003],  and  Vswm  [A(2,l  1)  =  8.397,;?  =  .009]. 
In  addition,  certain  requirements  and  assumptions  must  be  met  and  or  addressed  for  the 
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ANOVA  to  be  useful.  The  use  of  the  ANOVA  is  based  on  the  assumptions  that  the  SA 
distribution  follows  a  normal  curve,  equal  variances  are  present  across  conditions,  and  the 
observations  are  independent.  To  address  the  assumption  of  normality  of  the  SA 
distribution  and  homogeneity  of  variance  (homoscedasticity),  the  normality  of  the 
distribution  of  the  dependent  variable  data  must  be  examined  first  to  ensure  it  is 
parametric  (Dover,  1979).  This  was  accomplished  and  pointed  toward  a  very  small 
negative  skewness  =  -  0.029  and  kurtosis  (K  =  -0.153)  of  the  frequency  distribution  of  the 
sample  and  this  was  and  is  very  close  to  zero,  indicating  a  normal  distribution  (see 
Appendix  B)  (Dover,  1979).  Additionally,  a  z-score  was  computed  to  verify  the  critical 
value  of  ±  1.96  and  if  normality  significantly  exceeded  this  value  (Tabachnick  &  Fidell, 
2007).  To  determine  the  critical  value,  the  skew  =  -0.029  was  divided  by  the  standard 
error  of  skew  =  0.637  and  as  a  result  produced  a  z  score  =  -0.45;  this  is  well  within  the 
/>=0.05  threshold  of  z=±l  .96,  therefore  skew  was  not  significant.  In  addition,  the  graph 
illustrates  that  the  distribution  was  evenly  distributed,  and  skewness  is  within  the  range  of 
±  14  (skewness  =  -0.029)  (Wuensch,  2005a),  and  kurtosis  is  within  the  range  of  ±  3  (K  =  - 
0.153)  and  is  very  close  to  zero  (Wuensch,  2005b)  (see  Appendix  B).  Consequently,  the 
distribution  is  parametric  and  the  Levenes’s  test  for  the  equality  of  variances  may  be 
used. 

Next,  homoscedasticity  was  tested  by  the  Levene’s  test  for  homogeneity  for  Vp,  Va, 
and  Vswm  after  sorting  each  of  these  independent  variables  into  three  equal  and  logical 
groups.  Therefore,  the  neurocognitive  factors  were  placed  into  consistent  and  plausible 
categorical  levels.  Homogeneity  of  variances  were  successfully  tested  for  the  SA 
distribution,  Vp  (Levene’s  statistic  =  0.339,  df  2,9,  p  =  0.721),  Va(Levene's  statistic  = 
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0.469,  df  2,9,  p  =  0.640),  and  Vswm (Levene's  statistic  =  0.382,  df  2,9,  p  =  0.693). 
Consequently,  the  null  hypothesis  cannot  be  rejeeted,  therefore  the  groups  have  equal 
varianees  and  are  homogeneous,  henee  the  existenee  of  homoseedasticity.  For  this  study, 
the  observations  were  independent  and  the  samples  as  random  as  feasibly  possible  (near¬ 
random)  given  the  nature  of  this  field  experiment.  The  samples  were  indeed  from  the 
population  of  ANG  pilots  in  Ohio  and  were,  therefore,  representative.  Additionally,  the 
value  of  one  observation  was  not  related  to  any  other  observation  and  none  of  the 
subjeet’s  seores  provided  any  clues  as  to  how  any  of  the  other  subjeets  should  seore  nor 
were  any  of  events  dependent  on  another.  Consequently,  the  observations  were 
independent. 

Moreover,  the  additional  assumptions  of  multiple  regression  analysis  of  linearity, 
singularity,  multicollinearity,  absenee  of  outliers,  and  statistical  independence  of  the 
errors  were  also  verified.  The  majority  of  the  varianee  in  situation  awareness  is 
aeeounted  for  by  the  multiple  regression  model  eonsisting  of  Vp,  Va,  and  Vswm,  R  = 
0.753,  p  =  .008,  with  a  ealeulated  Durbin-Watson  statistie  of  1.425.  This  is  a  value  near 
2.0,  whieh  indieates  non-autoeorrelation.  Had  it  been  closer  to  0  or  4.0  it  would  show 
positive  autoeorrelation  or  negative  autoeorrelation,  respeetively.  The  residuals  are  not 
autoeorrelated.  Together,  these  validate  the  assumptions  of  linearity  and  statistieal 
independenee  of  the  errors.  Additionally,  multieollinearity  artificially  inflates  the 
varianees  of  the  parameter  estimates;  therefore,  this  eould  lead  to  the  lack  of  statistical 
significance  of  the  individual  predietor  variables  even  though  the  overall  model  may  be 
signifieant  (Tabaehniek  &  Fidell,  2001).  The  Varianee  Inflation  Faetor  (VIF)  quantifies 
the  severity  of  multieollinearity.  The  VIF  is  an  index,  whieh  measures  how  mueh 
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variance  of  the  estimated  regression  eoeffieient  is  inereased  beeause  of  multieollinearity. 
If  any  of  the  VIF  values  exeeed  five  (5.0),  it  could  be  inferred  that  the  assoeiated 
regression  eoeffieients  are  poorly  estimated  beeause  of  multieollinearity  (Montgomery, 
2001).  Vp  (VIF  =  1.504  <  5),  Va(VIF  =  1.246  <  5),  and  Vs™  (VIF  =  1.647  <5). 
Consequently,  multieollinearity,  was  and  is  not  an  issue  with  the  study  results,  the 
assumption  of  a  non-  multieollinearity  remains  valid.  If  these  violations  had  oecurred, 
the  predietive  variables  eould  have  beeome  redundant  with  eaeh  other,  and  in  that  ease 
one  predietive  variable  would  not  add  any  analytieal  significanee  over  another 
(Tabaehniek  &  Fidell,  2001).  This  did  not  oeeur.  Furthermore,  the  SPSS  will  exelude  or 
not  allow  a  variable  of  low  tolerance  to  enter  the  regression  model.  If  a  singularity  was 
present,  SPSS  would  not  have  eompleted  the  multi -regression  analysis  and  would  have 
provided  error  messages  in  the  results  matrix.  In  addition,  if  multieollinearity  had 
existed,  the  results  would  be  unstable  and  the  assumption  would  be  violated. 

Furthermore,  the  independent  variables  would  beeome  redundant  with  the  other  predietor 
variables  in  the  regression  model.  This  did  not  oeeur.  This  study  did  not  violate 
singularity  assumptions. 

For  the  assumption  of  the  absenee  of  outliers  and  more  on  the  linearity 
assumption,  a  normal  probability  plot  of  a  regression  for  SA  was  eonstrueted  to  observe 
the  data  distribution  for  Vp,  Va,  and  Vs™  (see  Appendix  B).  This  was  indieative  of  the 
normal  line  value  for  the  distribution  and  in  appearanee;  the  distributions  are  preeisely 
linear  with  no  prominent  outliers  present.  The  normal  probability  plot  analyses  in  this 
study  showed  a  linear  line  from  the  upper  left  to  lower  right  indieating  a  normal 
distribution  of  the  eriterion  dependent  variable  seores  of  SA  as  a  funetion  of  Vp,  Va,  and 


Ill 

Vswm  (Tabachnick  &  Fidell,  2001).  The  quasi-experimental  quantitative  method  and 
statistical  design  did  not  violate  this  assumption. 

Furthermore,  the  assumptions  of  normality,  homoscedasticity,  and  linearity  were 
not  violated  in  this  study.  Scores  (i.e.,  neurocognitive  predictor  variables)  in  the  study 
were  examined  for  probable  issues  that  could  affect  the  homoscedasticity  as  >  2  SD  from 
the  mean.  To  examine  if  any  issues  affected  the  distribution,  a  Cook’s  Di  (i.e.,  distance) 
statistical  analysis  was  computed.  Cook’s  Di  value  range  begins  at  zero  and  as  Di 
becomes  larger;  more  issues  may  become  significant  and  influence  the  linear  regression 
results.  The  Di  threshold  point  is  Di  >  1  or  more  conservatively  Di  >  0.078  (Cook, 

1977).  The  results  indicated  that  the  Cook’s  Di  =  0.001  {SD  =  0.824).  As  a 
consequence,  no  issues  were  not  significant  and  did  not  influence  the  results;  therefore, 
all  the  data  were  included  in  the  analyses.  The  study’s  statistical  data  analysis 
assumptions  were  not  violated  in  the  ANOVA  tests,  multiple  regression  assumptions,  and 
the  Pearson’s  product-moment  correlations.  In  the  following  sections,  the  demographic 
sample  is  described  along  with  the  statistical  procedures  used  to  analyze  the  research 
questions  and  tests  the  hypotheses. 

Results 

The  demographic  characteristics  are  presented  first  for  the  total  participant  sample 
(N  =  19).  The  demographic  characteristics  examined  in  the  study  were  age,  gender,  and 
education  level  at  testing.  Additionally,  all  19  subjects  where  highly  trained  and 
experienced  USAF  pilots  that  were  fully  qualified  in  the  C-27J  and  all  had  over  1,000 
hours  of  military  flight  time.  Following  the  demographic  characteristics,  the  hypotheses 
and  research  questions  are  discussed.  Participants  were  not  divided  into  groups,  but 
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instead  performed  repeated  trials  in  all  conditions  repeated  a  minimum  of  four  times  and 
this  contributed  to  a  reduction  in  the  subject  pool  to  twelve.  Specifically,  a  total  of  7 
subjects  dropped  out  of  the  study  over  time,  either  because  of  transfer  to  another  base  or 
election  to  not  participate  in  all  aspects  of  the  study. 

Age,  The  age  range  of  the  sample  was  3 1  to  47  years.  Figure  6  shows  the 
frequency  of  the  demographic  characteristics  of  the  group  age  in  the  study  population. 
The  modal  ages  were  35  and  46-47  (both  n  =  4),  which  together  comprised  50%  of  the  16 
participants  that  answered  the  age  question.  The  least  frequent  ages  were  31,  32,  and  39- 
45  (n  =  1)  (see  Figure  6). 


Age 


Figure  6.  Age  Demographics. 
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Gender,  The  sample  was  comprised  of  19  participants  of  whom  the  minority  was 
one  female  participant  and  the  majority  were  18  male  participants.  The  gender  ratio  was 
18:1;  while  highly  skewed  this  is  consistent  with  the  population  of  Air  National  Guard 
pilots  (USAF,  2007). 


Education 

Figure  7.  Education  Demographics. 

Education  Levels,  The  education  level  range  of  the  sample  was  from  Bachelors’ 
Degree  to  Masters’  plus  degree.  Figure  7  shows  the  frequency  of  the  demographic 
characteristics  of  education  levels  in  the  study  sample.  The  most  frequent  education  level 
was  Bachelor’s  Degree,  which  comprised  70%  of  the  participants  (n  =  13).  Two 
participants  had  Masters’  degrees,  and  one  had  a  Masters’  plus  (see  Figure  7). 
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Table  8 

Descriptive  Statistics  of  Demographics 


Descriptive  Statistics  of  Demographics 


Range 

Mean 

Std.  Dev 

N 

Age  (yrs) 

31-47 

38.15 

5.41 

16 

Education 

Bach  -  Master-i- 

Bach* 

N/A 

16 

Lemale 

1 

N/A 

N/A 

19 

Male 

18 

N/A 

N/A 

19 

*Mode 

Table  8  illustrates  the  resultant  demographies  for  Age,  Edueation  Level,  and  Gender. 

It  was  determined  that  during  analysis  it  was  best  to  use  a  eomposite  SA  seore 
that  eonsisted  of  eombining  both  the  China  Lake  and  Bedford  self-report  survey  seores 
(Bedford  Workload  Seale,  Roseoe,  &  Ellis,  1990;  China  Lake  Situation  Awareness  Seale, 
Adams,  Kane,  &  Bates,  1998)  and  taking  the  average  with  equal  weights  produeing  a 
more  sensitive  overall  self-report  SA  seore,  followed-up  by  using  the  objeetive 
psyehophysiologieal  measurements  to  validate  and  verify  the  eomputed  eomposite  SA 
seore.  Moreover,  this  best  measures  this  dissertation’s  derived  definition  of  situation 
awareness  which  is  the  psychological  ability  and  capacity  to  perceive  information  and  act 
on  it  acceptably.  Consequently,  this  was  verified  by  the  corresponding  HR  and  HRV 
objective  measurements  on  each  of  the  12  subjects  and  the  four  subjects  that  additionally 
had  reliable  EEG  data  (see  Table  13  and  Ligures  8,  9,  10,  and  1 1).  The  China  Lake 
survey  was  initially  derived  from  the  Bedford  survey  and  therefore  both  are 
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metaphorically  familial  and  extremely  similar  (Banbury,  Andre,  &  Croft,  2000).  The 
only  differences  are  in  the  way  the  questions  are  asked  -  in  effect,  they  are  alternate  forms 
of  the  same  questions.  For  example  the  China  Lake  asks  the  pilot  to  self-assess 
knowledge  level  (ability)  of  aircraft  state,  tasks  (if  task  shedding  needs  to  occur),  and  or 
environment,  whereas  the  Bedford  asks  the  pilot  to  self-assess  capacity  to  have 
knowledge  of  aircraft  state,  tasks  (if  task  shedding  needs  to  occur),  and  or  environment. 
All  subjects  responded  between  a  one  and  five  on  both  surveys  for  all  events  throughout 
varied  task  difficulty.  One  through  five  are  a  nearly  a  one-to-one  relationship  between 
the  surveys,  however,  it  was  noticed  that  even  though  the  subject  would  answer  both 
surveys  very  similarly,  repeatedly  every  subject  would  answer  the  China  Lake  one 
interval  above  the  Bedford  for  the  same  identical  tasks.  It  was  observed  that  the  pilot 
subjects  would  underreport  their  SA  difficulty  and  overinflate  their  knowledge  of  aircraft 
state  on  the  China  Lake,  likely  due  to  the  fact  it  contained  in  the  title  the  words  “Situation 
Awareness  Level”.  The  pilots  tended  to  have  a  perceived  pressure  to  fully  handle  all 
challenges  and  avoid  triggering  additional  follow-up  if  they  admitted  to  having  “bad  SA” 
whereas,  they  were  more  forthcoming  in  describing  their  workload  level.  It  was, 
therefore,  more  apropos  to  using  a  Composite  SA  score  between  the  two  surveys  that  was 
verified  by  corresponding  HR  and  HRV  objective  data.  Additionally,  the  researcher 
observed  that  during  high  task  difficulty  the  pilot  would  tend  to  commit  minor,  non¬ 
safety  critical  errors  such  as  missing  some  of  the  radio  calls,  overshooting  altitude  level 
offs,  target  airspeeds,  and  generally  flying  a  less  stable  platform.  In  these  situations,  the 
pilot  would  be  more  forthcoming  in  their  self-assessment  on  the  Bedford  mental 
workload  scale  but  less  forthcoming  on  their  self-assessment  on  the  China  Lake  situation 
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awareness  scale,  when  logically  both  surveys  should  have  been  a  one-to-one  and  equal 
type  relationship.  However,  repeatedly  the  pilot  rated  his  or  her  own  SA  as  one  interval 
higher  than  their  own  workload,  consequently  applying  equal  weights  to  both  surveys  and 
then  taking  the  average  for  a  composite  SA  score  was  the  most  conservative  and 
reasonable  solution  to  maximizing  the  validity  and  sensitivity  of  the  SA  assessment. 


Figure  8.  Photograph,  ensuring  electrical  conductivity  during  EEG  Eeads  and  Cap 
Installation  of  EEG  before  flight. 
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Figure  9.  Photograph,  ECG  Lead  Installation  (pre-flight). 

SA,  HR,  HRV,  Vp,  Va,  and  Vswm. 

Table  9 


Descriptive  Statistics  of  Variables 


Descriptive  Statistics  of  Variables 


Range 

Mean 

Std.  Dev 

N 

SA  (Composite  Seore)  Interval  Level 

1.875-3.0 

2.4238 

0.31492 

12 

HR  (bpm)  Ratio  Level  = 

76.97-  102.27 

+  25.3 

N/A 

12 

HRV  (ms)  Ratio  Level  A 

71.63  -45.17 

-26.46 

N/A 

12 

Vp  (Quotient  Score)  Ratio  Level 

28-119 

88.4167 

26.92568 

12 

Va  (Quotient  Score)  Ratio  Level 

45-128 

93.2500 

23.38269 

12 

Vswm  (percentage)  Ratio  Level 

50-98 

81.6667 

15.39382 

12 

Table  9  illustrates  study  results  for  SA,  HR,  HRV,  Vp,  Va,  and  Vswm. 
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Figure  10.  Typical  HR  increase  and  HRV  decrease  as  composite  SA  score  changes 
(China  Lake  and  Bedford  increase  [SA  is  going  down  and  WL  is  going  up]). 


Typical  HR  Increase  and  HRV  Decrease  as  SA  Decreases, 
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Heart  Rate  ^  ariability,  HDD  vs.  HUD 
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Figure  11.  Average  SDNN  for  instrument  approaehes  eondueted  heads  down  (HDD) 
versus  heads-up  display  (HUD).  Bar  height  represents  the  mean,  and  error  bars  are  one 
standard  error  of  the  mean.  Note  that  lower  values  are  assoeiated  with  deereased  SA  and 
inereased  WL. 

HRV:  HDD  vs  HUD. 

Inter-beat  intervals  were  exported  for  subsequent  analyses  of  average  HR  and  SDNN  (in 
this  ease,  standard  deviation  of  the  R-R  interval)  by  way  of  Fourier  analysis  using 
MATLAB  (see  Table  9,  Figure  9,  Figure  10,  11,  12,  and  Table  10). 
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Figure  12.  Illustrating  R-R  Interval  (Wilson,  2002). 


The  Fourier  transform  was  used  in  MATLAB  to  obtain  the  HRV  spectrum. 

X(,f)=\'  x(i)  e--‘-dl 

f{x)  =  ^]F{co)e-dco 

£71  -ac 


F{o))  =  J/(.v)e-“* 


Specifically,  the  Fast  Fourier  Transforms  were  applied  to  the  IBI  sequence.  Additionally 
they  were  used  for  EEG  data  analysis. 
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Figure  13.  EEG  variable. 


Circled  areas  in  Eigure  13  illustrate  increased  workload  and  lowered  SA.  This  is  where 
Pz  Alpha  and  Ez  Theta  brainwaves  (electromagnetic  energy)  diverge  in  electrical  power. 
Note,  that  in  the  top  plot,  while  using  the  HDD  on  the  VER  final  approach  and  landing 
the  pilot  experienced  an  increase  in  mental  workload  and  a  decrease  in  cognitive  SA. 

P5’s  Pz  Alpha  and  Ez  Theta  brainwaves  diverge  most  notably,  while  P5  also  experienced  a 
drop  in  HRV  of  31.4  ms,  while  at  the  same  time  P5’s  composite  SA  score  went  from  a 
score  of  2  to  3. 


Pz  Alpha  and  Fz  Theta  Brainwaves  Diverge  as  SA  is  Reduced. 


138 


For  Entire  Flight: 
Table  10 
Correlations 


Correlations 

SA 

Vp 

Va 

Vswm 

Pearson  Correlation  SA 

1.000 

-.816 

-.402 

-.705 

Vp 

-.816 

1.000 

.331 

.571 

Va 

-.402 

.331 

1.000 

.433 

Vswm 

-.705 

.571 

.433 

1.000 

Sig.  (2-tailed)  SA 

.002 

.048 

.01 

Vp 

.002 

.147 

.026 

Va 

.048 

.147 

.080 

Vswm 

.01 

.026 

.080 

N  SA 

12 

12 

12 

12 

Vp 

12 

12 

12 

12 

Va 

12 

12 

12 

12 

Vswm 

12 

12 

12 

12 

SA  =  Dependent 
Variable 


Correlations 

Overall 

Hard 

Easy 

Vp 

-0.816 

Va 

-0.583 

Pearson  Correlation 

Vswm 

-0.634 

Vp 

0.002 

Va 

0.046 

Sig.  (2 -tailed) 

Vswm 

0.026 

Vp 

12 

12 

12 

Va 

12 

12 

12 

N 

Vswm 

12 

12 

12 

Table  10  illustrates  the  Pearson  Correlation  between  the  variables  (SA,  Vp,  Va,  and  Vswm) 
as  task  diffieulty  varied,  the  2-tailed  signifieance  levels,  andN-the  number  of  subjects. 
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Table  1 1  Factorial  ANOVA  -  Repeated  Measures  within  Subjects. 


For  the  Overall  Sorties 

Dependent  Variable:  SA 


Type  111  Sum 

Source 

of  Squares 

df 

Mean  Square 

F 

Sig. 

Corrected 

Model 

1.087" 

9 

.121 

71.106 

.014 

Intercept 

51.844 

1 

51.844 

30508.734 

.000 

Vp 

.197 

2 

.098 

57.875 

.017 

Va 

.060 

2 

.030 

17.729 

.053 

Vswm 

.006 

2 

.003 

1.807 

.356 

Vp*  Va 

.000 

1 

.000 

.182 

.711 

Vp  *  Vswm 

.007 

1 

.007 

4.160 

.178 

Va  *  Vswm 

.000 

0 

V  *  V  *  V 

.000 

0 

Error 

.003 

2 

.002 

Total 

71.586 

12 

Corrected  Total 

1.091 

11 

a.  R  Squared  =  .997  (Adjusted  R  Squared  =  .983) 


Table  1 1  illustrates  the  Faetorial  ANOVA  -  Repeated  Measures  within  Subjects  with  SA 
as  the  dependent  variable. 
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Table  12 
ANOVA 


ANOVA" 

Model 

Sum  of  Squares 

df 

Mean  Square  F 

Sig. 

Regression 

.821 

3 

.274  8.125 

.008’’ 

Residual 

.270 

8 

.034 

Total 

1.091 

11 

a.  Dependent  Variable:  SA 

b.  Predietors:  (Constant),  Vswm,  Va,  Vp 

Table  12  illustrates  that  the  multiple  regression  within  faetors  and  repeated 
measures  ANOVA  was  signifieant  for  the  predietors’  (Independent  Variables: 
Vswm,  Va,  and  Vp)  effeets  on  the  eriterion  (Dependent  Variable)  SA,  p  <  0.05. 


Table  13 
Model  Summary 


Model 

Summary 

Change  Statistics 

R 

Adjusted 

Std.  Error 
of  the 

R 

Square  F 

Sig.F 

R  Square 

R  Square 

Estimate 

Change 

Change  dfl  df2 

Change 

.868"  .753 

.660 

.18356 

.753 

8.125  3  8 

.008 

a.  Predictors:  (Constant),  Vswm,  Va,  Vp 


Table  13  illustrates  that  together  Vswm,  Va,  Vp,  aeeount  for  75.3  pereent  of  the  varianee  in 
SA,  p<0.05. 
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Table  14 

Coefficients  Dependent  Variable:  SA 


Coefficients  Dependent 
Variable:  SA 


Entire  Flight 

Easy  Task 

Difficult  Task 

Standardized 

Coefficients 

Correlations 

Standardized 

Coefficients 

Correlations 

Standardized 

Coefficients 

Correlations 

Visual 

Ability 

P 

Part 

P 

Part 

P 

Part 

Va 

-0.057 

-0.051 

-0.229 

-0.186 

-0.460 

-0.375 

Vp 

-0.606 

-0.494 

+0.062 

+0.055 

-0.277 

-0.248 

V  swm 

-0.334 

-0.26 

-0.530 

-0.413 

-0.055 

-0.042 

Table  14  illustrates  the  ^-standardized  eoeffieient  and  Part-eorrelation,  for  Va,  Vp,  and 
Vswm  as  predietors  for  SA  for  the  Entire  Flight,  during  Easy  Task,  and  Diffieult  Task. 


For  the  3-way  faetorial  ANOVA,  for  the  overall  sortie  Vp  [F(2,  11)=  57.875, = 
.017],  however,  there  were  no  signifieant  effeets  between  Vp,  Va,  and  or  Vswm  (see 
Table  1 1).  This  shows  that  Vp  performed  the  most  important  role.  Additionally,  for  the 
overall  sortie  Va  [F{2,  11)=  17.729, =  .053],  even  though  not  statistieally  signifieant,  it 
illustrates  that  Va  played  a  key  role  for  the  overall  sortie  even  though  subordinate  to  Vp. 

It  should  also  be  noted  that  Vp  and  Vswm  had  although  not  a  signifieant  interaetion,  were 
interaeting  together  more  so  than  any  other  eombination  Vp  *  Vswm  [F(l,  11)=  4.160,  p 
=  .178].  This  is  in-line  with  the  multiple  regression  analyses  and  modeling  that  was  also 
performed  for  the  overall  sortie  as  well,  in  addition  to  the  varied  task  difficulty  results 
(Easy  versus  Difficult). 

Furthermore,  the  tests  were  statistically  significant  for  the  predictor  Vp,  as  task 
difficulty  was  varied,  F(2,l  1)  =  \2A25,p=  .003.  Post-hoc  comparisons  using  the 
Scheffe  test  indicated  that  pilots  with  high  Vp  scores  had  reliably  higher  SA  scores  than 
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pilots  who  had  low  Vp  scores  {p  =  0.006).  However,  pilots  with  medium  Vp  scores  had 
SA  scores  that  did  not  significantly  differ  from  the  high  or  low  Vp  scores.  Taken 
together,  these  results  suggest  that  high  levels  of  Vp  do  have  an  effect  on  SA,  as  do  low 
scores.  The  tests  were  statistieally  significant  for  the  predietor  Va,  during  demanding 
levels  of  task  diffieulty,  F(2,l  1)  =  5.749, =  .025.  Post-hoc  comparisons  using  the 
Seheffe  test  indieated  that  pilots  with  high  Va  scores  had  reliably  higher  SA  scores  than 
pilots  who  had  low  Va  scores  {p  =  0.009).  However,  pilots  with  medium  Va  scores  had 
SA  seores  that  did  not  signifieantly  differ  from  the  high  or  low  Va  seores.  Taken 
together,  these  results  suggest  that  high  levels  of  Va  do  have  an  effeet  on  SA,  as  do  low 
scores.  Furthermore,  the  tests  were  statistieally  signifieant  for  the  predietor  Vswm,  during 
easy  levels  of  task  diffieulty,  F(2,l  1)  =  8.397, =  .009.  Post-hoe  comparisons  using  the 
Seheffe  test  indieated  that  pilots  with  high  Vswm  scores  had  reliably  higher  SA  seores  than 
pilots  who  had  low  Vswm  scores  {p  =  0.007).  However,  pilots  with  medium  Vswm  scores 
had  SA  scores  that  did  not  significantly  differ  from  the  high  or  low  Vswm  scores.  Taken 
together,  these  results  suggest  that  high  levels  of  Vswm  do  have  an  effeet  on  SA,  as  do  low 
scores. 

It  was  diseovered  that  for  this  researeh  that  placing  visual  abilities  (a  ratio  level 
variable)  into  3-bins  as  a  function  of  Z-score  in  order  to  eategorize  them  as  Low, 

Medium,  and  High  (a  nominal  variable),  that  while  redueing  nuisance  error,  created  a 
small  loss  of  fidelity.  High  aeeuraey  was  needed  in  order  to  diseover  the  exact 
contributions,  relationships,  and  predictability  of  these  three  neurocognitive  abilities  Va, 
Vp,  and  Vswm.  Henee,  therefore,  in  order  to  maintain  preeise  granularity,  multiple 
regression  within  factors  and  repeated  measures  analysis,  and  the  Pearson  correlation 
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were  additionally  utilized  for  hypothesis  testing  and  model  generation.  Allowing  the 
ecological  validity  and  exactitude  of  this  study  to  be  maximized. 

Testing  of  Hypotheses 

The  following  section  is  an  investigation  of  the  study  research  questions  and 
hypotheses  through  ANOVA  tests  to  test  the  null  hypotheses.  Correlations  and  multiple 
regressions  were  computed  in  the  design  analyses  to  support  the  testing  of  the 
hypotheses.  The  results  are  presented  along  with  the  ANOVA  tests  for  each  hypothesis 
to  determine  the  significance  between  the  mean  scores.  The  research  questions  and 
hypotheses  were  focused  on  situation  awareness  predictor  variables.  The  Pearson 
correlation  results  are  reported  in  Table  12  for  the  independent  variables  to  determine  the 
relationships  with  situation  awareness  scores.  In  addition,  regressions  were  conducted  to 
provide  statistical  tests  in  support  of  hypotheses  1,  2,  and  3.  Multiple  regression  analyses 
were  computed  to  determine  the  contributions  of  visual  attentiveness,  visual 
perceptiveness,  and  visual  spatial  working  memory  to  situation  awareness.  The  three 
research  questions  and  hypotheses  of  the  relationships  and  predictors  of  situation 
awareness  are  addressed  through  the  data  analyses  and  results  presented  below. 

This  section  contains  the  results  for  the  research  questions  and  hypotheses: 

Ql.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of  subject 
inherent  visual  attentiveness  on  situation  awareness? 

Hlo.  Under  varying  task  difficulty,  there  will  be  no  statistically  significant  effects  of 
inherent  visual  attentiveness,  (as  measured  by  the  Integrated  Visual  and  Auditory 
Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on  resulting  situation 


awareness. 


Hla.  Under  varying  task  difficulty,  there  will  be  statistically  significant  effects  of 
inherent  visual  attentiveness,  (as  measured  by  the  Integrated  Visual  and  Auditory 
Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on  resulting  situation 
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awareness. 

For  Ql,  the  null  hypothesis  was  rejected.  The  tests  were  statistically  significant  for  the 
predictor  visual  ability  Va  as  task  difficulty  was  varied;  F(2,l  1)  =  5.749,  p  <  0.025.  In 
addition,  multiple  regression  analyses  revealed  that  for  this  model  {R  =  0.753,  p  =  .008). 
During  high  task  difficulty  Va  {r{\2\  =  -0.583, />  =  .023)  had  the  strongest  relationship. 
Post-Hoc  tests  revealed  a  Cohen’s  =  3.05  yielding  statistical  power  to  be  0.98. 

Q2.  Under  varying  task  difficulty,  are  there  statistically  significant  effects  of  subject 
inherent  visual  perceptiveness  on  situation  awareness? 

H2o.  Under  varying  task  difficulty,  there  will  be  no  statistically  significant  effects  of 
inherent  visual  perceptiveness,  (as  measured  by  the  Integrated  Visual  and  Auditory 
Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on  resulting  situation 
awareness. 

H2a.  Under  varying  task  difficulty,  there  will  be  statistically  significant  effects  of 
inherent  visual  perceptiveness,  (as  measured  by  the  Integrated  Visual  and  Auditory 
Continuous  Performance  Test  Plus  [IV A+];  Brain  Train,  2010),  on  resulting  situation 
awareness. 

For  Q2,  the  null  hypothesis  was  rejected.  The  tests  were  statistically  significant  for  the 
predictor  visual  abilities  Vp,  as  task  difficulty  was  varied;  F(2,l  1)  =  \2.\25,p=  .003.  In 
addition,  multiple  regression  analyses  revealed  that  for  this  model  {R  =  0.753,  p  = 

.008;  Predicted  Z^a  —  11.646  —  0.606Z^.p  —  O.OSlZya  —  0.334Zyg^^).  Moreover, 
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the  Pearson  eorrelation  results  indieated  that  Vp  (r[12]  =  -0.816, =  .001)  had  the 
strongest  relationship  of  the  three  neuroeognitive  faetors  for  the  overall  sortie.  Post-Hoe 
tests  revealed  a  Cohen’s  =  3.05  yielding  statistieal  power  to  be  0.98. 

Q3.  Under  varying  task  diffieulty,  are  there  statistieally  signifieant  effeets  of  subjeet 
inherent  visual  spatial  working  memory  on  situation  awareness? 

H3o.  Under  varying  task  diffieulty,  there  will  be  no  statistieally  signifieant  effects  of 
Visuospatial  Working  Memory  Test  results  (AFRL,  2009),  on  resulting  situation 
awareness. 

H3a.  There  will  be  statistically  significant  effects  of  Visuospatial  Working  Memory  Test 
results  (AFRL,  2009),  on  resulting  situation  awareness. 

For  Q3,  the  null  hypothesis  was  rejected.  The  tests  were  statistically  significant  for  the 
predictor  visual  ability  Vswm  as  task  difficulty  was  varied;  F(2,l  1)  =  8.397,  p  =  .009.  In 
addition,  multiple  regression  analyses  revealed  that  for  this  model  {R  =  0.753,  p  =  .008). 
Post-Hoc  tests  showed  a  Cohen’s  =  3.05  yielding  Statistical  power  to  be  0.98. 
Interestingly  during  low  task  difficulty  Vswm  (^[12]  =  -0.634, /»  =  .013)  had  the  strongest 
relationship. 

Evaluation  of  Findings 

This  study  was  an  assessment  of  the  extent  to  which  visual  attentiveness,  visual 
perceptiveness,  and  visuospatial  working  memory  had  an  effect  and  relationship  with  SA. 
Based  on  the  findings  presented  in  this  study,  it  is  evident  that  these  cognitive  abilities  are 
significant  predictors  of  SA,  explaining  in  total  75%  of  the  variance  in  SA.  The  findings 
supported  the  first  alternative  hypothesis  (Hi a)  by  demonstrating  a  statistically  significant 
effect  and  relationship  between  visual  attentiveness  and  SA.  Furthermore,  the  results 
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supported  the  second  alternative  hypothesis  (H2a)  in  that  there  was  a  statistically 
significant  effect  and  relationship  between  visual  perceptiveness  and  SA,  and  in  relation 
to  the  third  alternative  hypothesis  (H3a)  there  was  a  statistically  significant  effect  and 
relationship  between  Visual  Spatial  Working  Memory  and  SA.  The  correlation  and 
regression  analyses  results  provide  further  strong  support  for  a  statistically  significant 
relationship  between  these  predictors  and  SA. 

Visual  attentiveness,  visual  perceptiveness,  and  visuospatial  working  memory 
were  predictors  of  SA.  Visual  Perceptiveness  (Vp)  is  the  ability  to  have  perceptivity,  to 
be  insightful,  and  to  have  discernment  -  the  ability  to  perceive  that  which  is  obscured. 

This  variable  is  derived  from  visual  Prudence,  Consistency,  and  Stamina  scales  (Sandford 
&Tumer,  2009).  The  study  results  indicated  that  Vp  (r[12]  =  -0.816,/?  =  .001)  had  the 
strongest  relationship  of  the  three  neurocognitive  factors  for  the  overall  sortie.  This 
indicates  that  visual  perceptiveness  is  the  most  important  trait  for  situation  awareness 
(SA).  During  a  typical  sortie,  task  difficulty  will  range  from  low  to  high  and  in-between, 
therefore  for  a  typical  sortie  Vp  is  the  most  important  and  most  influential  factor,  variable, 
and  predictor  overall  of  S  A. 

Prudence  is  a  measure  of  impulsivity  and  response  inhibition.  A  higher  Prudence 
score  is  indicative  of  traits  consisting  of  being  circumspective  and  mindful.  The  IV A+ 
measures  three  types  of  commission  errors  —  impulsivity,  propensity,  and  mode  shift  to 
obtain  the  Prudence  score  =  100  -  ((Number  of  Prudence  visual  errors  /  65)  *  100, 
Sandford  &Tumer,  2009).  Descriptors  of  Prudence  are  Focused  Attention  and  Selective 
Attention.  Consistency,  the  second  major  factor  for  Vp,  is  sustaining  a  reliable  effort  and 
staying  on  task.  Therefore,  Consistency  as  a  measure  of  the  reliability  and  variability  of 
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response  times  =  (Quartile  1  visual  reaction  time  /  Quartile  3  visual  reaction  time  *  100, 
Sandford  &Tumer,  2009).  The  third  major  factor  is  Stamina,  and  this  is  being  able  to 
maintain  speed  of  mental  processing.  Stamina  compares  the  mean  reaction  times  of 
correct  responses  during  the  first  200  trials  to  the  last  200  trials  and  equals  =  ((Mean 
visual  reaction  time  (MVRT)  of  sets  1+2)  /  (MVRT  of  sets  4+5))  *  100  (Sandford 
&Tumer,  2009).  The  Vp  score  was  a  composite  quotient  score  that  was  derived  from 
these  three  categories  by  means  of  the  IVA  +  software. 

Visual  Attentiveness  (Va)  is  the  ability  to  concentrate  and  be  devoted,  to  be 
diligent  and  detailed,  alert,  watchful,  and  responsive.  This  variable  is  derived  from  visual 
Vigilance,  Focus,  and  Speed  scales  (Sandford  &Tumer,  2009).  The  study  results 
indicated  that  Va  (^[12]  =  -0.583, =  .023)  had  the  strongest  relationship  of  the  three 
neurocognitive  factors  during  high  task  difficulty.  This  indicates  that  visual  attentiveness 
is  the  most  important  trait  during  high  task  difficulty  for  situation  awareness  (SA). 
Consequently,  during  a  phase  when  task  difficulty  is  high,  Va  will  rise  to  the  top  as  the 
most  important  and  most  influential  factor,  variable,  and  predictor  of  SA. 

Vigilance  is  a  measure  of  inattention  as  evidenced  by  two  different  types  of  errors 
of  omission.  One  error  is  defined  as  a  failure  to  respond  to  a  target  and  the  other  is 
defined  as  a  non-response  to  a  target  immediately  after  a  foil  has  been  presented  during 
frequent  blocks.  A  higher  Vigilance  score  is  indicative  of  traits  consisting  of  being 
intense  and  accurately  responsive.  The  1VA+  Va  =  100  -  ((Number  of  Vigilance  visual 
errors  /  45)  *  100)  (Sandford  &Turner,  2009).  To  be  vigilant,  an  individual  must 
maintain  and  direct  attentional  effort  to  classify  each  stimuli  as  either  a  target  or  foil  and 
then  make  the  appropriate  response  (Sandford  &Turner,  2009).  Focus  is  the  second 
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major  factor  for  Va  and  reflects  the  varianee  in  reaetion  time  speed  measuring  the  ability 
to  sustain  and  maintain  attention.  Therefore,  Foeus  =  1  -  (SD  of  visual  reaetion  times  / 
Mean  visual  reaction  time))  *  100  (Sandford  &Turner,  2009).  The  third  major  faetor  is 
Speed  reflects  the  average  reaetion  time  for  eorrect  trials  throughout  the  test,  thus  Speed 
measures  diseriminatory  mental  proeessing  speed.  Speed  equals  =  Mean  visual  reaction 
time  in  milliseconds  for  all  correct  trials  (Sandford  &Tumer,  2009).  The  Va  score  was  a 
composite  quotient  seore  that  was  derived  from  these  three  categories  by  means  of  the 
IVA  +  software. 

Visuospatial  Working  Memory  (Vswm)  is  the  ability  to  have  the  part  of  memory 
that  is  responsible  for  reeording  and  logging  information  about  one's  spatial  environment 
including  spatial  orientation  of  one-self,  eoupled  with  actively  holding  multiple  pieees  of 
transitory  spatial  information  in  the  mind,  where  it  can  be  manipulated.  This  variable  is 
derived  from  the  Visuospatial  Working  Memory  N-back  test  (AFRL,  2009;  Gevins  & 
Cutillo,  1993;  Sohn,  &  Doane,  1997,  2000,  2003).  The  study  results  indicated  that  Vswm 
(r[12]  =  -0.634, =  .013)  had  the  strongest  relationship  of  the  three  neuroeognitive 
faetors  during  low  task  diffieulty.  This  indieates  that  visuospatial  working  memory 
(Vswm)  is  the  most  important  trait  during  low  task  diffieulty  for  situation  awareness  (SA). 
Consequently,  during  a  phase  when  task  difficulty  is  low,  Vswm  will  rise  to  the  top  as  the 
most  important  and  most  influential  faetor,  variable,  and  predietor  of  SA. 

It  is  noteworthy  that  for  the  overall  entire  flight  Vswm  was  the  seeond  most 
influential  factor  and  predictor  of  SA  (see  Table  14).  Further  studies  should  be 
eondueted  to  validate  these  neuroeognitive  predictors  of  SA  within  diverse  populations 
that  take  into  eonsideration  age,  experienee,  and  edueation  level.  Even  with  the 
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limitations  of  the  study,  the  findings  ean  be  used  as  a  basis  for  examining  other  potential 
predictors  while  enhancing  the  current  TMSA;  additionally  this  study  improved  the 
definition  of  SA.  The  following  chapter  presents  and  discusses  the  implications  and 
recommendations  that  emerged  from  this  study. 
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Chapter  5:  Implications,  Recommendations,  and  Conclusions 

This  chapter  will  cover  a  brief  review  of  the  problem  statement,  purpose,  method, 
limitations,  and  ethical  dimensions.  Implications  will  be  discussed  as  well  as  theoretical 
and  practical  significance.  This  research  will  be  summarized  involving  questions  and 
hypothesis  in  order  to  draw  logical  conclusions.  Additionally  the  results  will  be  put  back 
into  context  by  describing  how  the  results  respond  to  the  study  problem,  fit  with  the 
purpose,  demonstrate  the  significance,  and  contribute  to  the  existing  literature  described 
in  Chapter  2.  Finally,  this  chapter  will  conclude  with  recommendations  and  conclusions 
(also,  see  Appendix  B). 

Cognitive  situation  awareness  has  been  a  topic  of  exploration  in  the  behavioral 
sciences.  Situation  awareness  may  be  modestly  defined  as  the  human  ability  to  perceive 
and  comprehend  the  environment,  and  use  that  information  to  undertake  a  task  (Endsley, 
1995a).  Previous  exploration  suggested  that  cognitive  situation  awareness  is  critical  to  a 
wide  range  of  human  performance,  but  most  notably  in  aviation  (Bailey,  Shelton,  & 
Arthur,  2011;  Cass,  2011;  Crawford  &  Neal,  2006;  Salmon,  et  ah,  2010;  Kim,  2009; 
Wickens,  2008).  In  aviation  from  1980  to  2007,  983  accidents  occurred  worldwide 
involving  aircraft  without  an  installed  Head  Up  Display  (HUD)  (Flight  Safety 
Foundation,  2009).  A  recent  review  of  this  accident  data  revealed  that  the  majority  were 
due  to  reduced  cognitive  situation  awareness  of  the  aviator,  combined  with  limited 
visibility,  and  operations  into  austere  locations  (Arthur,  Prinzel,  Williams,  &  Kramer, 
2006;  Bulkley,  Dyre,  Few,  &  Caufield,  2009;  Flight  Safety  Foundation,  2009;  NTSB, 
2009).  Despite  the  importance  of  situation  awareness  in  human  performance,  in  the  past 
there  has  been  no  consensus  on  predictors  or  antecedents  (it  has  been  unpredictable),  an 
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area  that  tended  to  be  elusive  and  theoretieally  vague  (Cass,  2011;  Douglas,  Aleva,  & 
Havig,  2007;  Dekker  &  Hollnagel,  2004;  Gorman,  Cooke,  &  Winner,  2006;  Harbour, 
Hudson,  &  Zehner,  2012;  Jodlowski,  2008;  Rousseau,  Tremblay,  &  Breton,  2004; 
Substyawati,  Wiekens,  &  Chui,  2011;  Wiekens,  2002,  2008).  A  better  understanding  of 
the  anteeedents  of  situation  awareness  will  advance  the  theory  of  how  situation  awareness 
is  formed  and  maintained,  as  well  as  produce  improvements  that  could  apply  to  any  task 
involving  visual  displays,  information  integration,  and  mental  training. 

The  focus  of  this  project  was  to  conduct  neuroergonomic  empirical  studies  that 
identified  the  primary  abilities  that  are  essential  antecedents  of  situation  awareness 
(causing  it  to  be  predictable).  As  the  scientific  concept  of  situation  awareness  must  be 
defined  with  respect  to  a  particular  task,  this  project  focused  on  the  complex  multitask  of 
piloting  a  military  aircraft.  To  reduce  the  confounds  of  variance  in  pilot  populations  and 
aircraft  types,  the  scope  was  confined  to  the  USAF  Air  National  Guard  population  who 
fly  tactical  airlift  in  north  central  Ohio  with  emphasis  on  task  saturating  phases  of  flight, 
where  situation  awareness  was  especially  stressed  due  to  increased  mental  workload 
(FSF,  2009).  It  should  enhance  behavioral  science  in  this  area  by  shedding  light  on  the 
significance  of  visual  attention  and  visuospatial  working  memory  as  the  antecedents  or 
predictors  of  cognitive  situation  awareness.  In  doing  so,  this  work  answered  the 
literature’s  exigent  call  to  advance  cognitive  situation  awareness  theory  (Endsley,  2012; 
Gutzwiller  &  Clegg,  2012;  Harbour  &  Hudson,  2013;  Substyawati,  Wiekens,  &  Chui, 
2011;  Vidulich  &  Tsang,  2012;  Wiekens  &  McCarley,  2008),  while  addressing  a  problem 
of  real-world  significance  (Bulkley,  Dyre,  Lew,  &  Canfield,  2009;  Flight  Safety 
Foundation,  2009;  NTSB,  2009;  Wiekens  &  McCarley,  2008). 
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There  was  limited  literature  and  research  regarding  neurocognitive  predictors  of 
situation  awareness  and  display  usability  with  USAF  pilots  while  performing  complex 
tasks  (Douglas,  Aleva,  &  Havig,  2007;  Ellis  &  Levy,  2009;  Endsley,  2012;  Gillan  et  ah, 
2009;  Gutzwiller  &  Clegg,  2012;  Harbour  et  ah,  2012;  Jen-li,  Ruey-Yun,  &  Ching-Jung, 
2013;  Jones,  Connors,  &  Endsley,  2011;  Sulistyawati,  Wickens,  &  Chui,  2011;  Vidulich 
&  Tsang,  2012;  Wickens,  2008;  Wickens  &  McCarley,  2008;  Tirre  &  Gugerty,  1999, 
2000).  Visual  processing  was  categorized  as  static  and  dynamic  (Proctor  &  Vu,  2010), 
and  operationalized  as  visual  attentiveness  and  perceptiveness,  integrated  with 
visuospatial  memory  (Brain  Train,  2010;  Christensen  et  ah,  2013;  Corbett  &  Constantine, 
2007;  Endsley,  2012;  Gugerty,  in  press;  Sandford,  Line,  &  Goldman,  1995;  Sandford  & 
Turner,  1994;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley,  2008). 

The  problem  that  was  addressed  by  this  research  was  that  despite  the  scientific 
exigency  there  were  no  precise  and  predictive  nexuses  for  current  theories  of  situation 
awareness  and  concrete  quantifiable  cognitive  and  perceptual  processes  (Douglas,  Aleva, 
&  Havig,  2007;  Ellis  &  Levy,  2009;  Elliott  et  ah,  2009;  Endsley,  2012;  Gillan  et  ah, 

2009;  Gugerty,  in  press;  Gutzwiller  &  Clegg,  2012;  Harbour  et  ah,  2012;  Jen-li,  Ruey- 
Yun,  &  Ching-Jung,  2013;  Jones,  Connors,  &  Endsley,  2011;  Lau,  Jamieson,  & 
Skranning,  2013;  Proctor  &  Vu,  2010;  Sulistyawati,  Wickens,  &  Chui,  2011;  Vidulich  & 
Tsang,  2012;  Wickens,  2008;  Wickens  &  McCarley,  2008;  Tirre  &  Gugerty,  1999, 

2000).  After  years  of  work  since  Endsley’ s  Theoretical  Model  of  Situation  Awareness 
(TMSA,  1995a)  was  first  published,  current  models  of  situation  awareness  remained 
conceptual  models  that  provided  little  specificity  with  regards  to  the  neurocognitive 
processes  that  are  necessary  for  the  formation  and  maintenance  of  situation  awareness 
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(Gillan  et  al.,  2009;  Lau,  Jamieson,  &  Skranning,  2013;  Vidulich  &  Tsang,  2012; 

Wickens  &  McCarley,  2008).  While  the  TMSA  reeognized  pereeption  as  a  eritical  first 
step,  there  were  no  specifie  or  quantitative  links  between  perceptual  abilities  and  situation 
awareness;  nor  did  subsequent  work  prior  to  this  dissertation  been  able  to  clarify  the 
issue,  e.g.  the  effects  of  visual  ability  on  the  level  of  situation  awareness  (Jen-li,  Ruey- 
Yun,  &  Ching-Jung,  2013;  Jones,  Connors,  &  Endsley,  2011;  Vidulich  &  Tsang,  2012; 
Sulistyawati,  Wickens,  &  Chui,  2011).  Needed  theoretical  advancement  in  this  area  was 
hampered  by  a  lack  of  specific,  testable  predictions  regarding  plausible  component 
processes;  there  has  been  little  theoretical  progress  due  to  this  (Douglas,  Aleva,  &  Havig, 
2007;  Endsley,  2012;  Gutzwiller  &  Clegg,  2012;  Harbour  et  ah,  2012;  Vidulich  &  Tsang, 
2012;  Wickens,  2008;  Sulistyawati,  Wickens,  &  Chui,  2011). 

The  seminal  research  done  by  Tirre  and  Gugerty  (1999,  2000),  Wickens  and 
McCarley  (2008),  Elliott  et  ah,  (2009),  Jen-li,  et  ah,  (2013),  and  Gugerty,  (in  press)  found 
that  visual  processing  (Proctor  &  Vu,  2010),  was  involved  in  situation  awareness 
(Vidulich  &  Tsang,  2012).  However,  the  specific  neurocognitive  component  processes 
had  not  been  explored  in  detail  prior  to  this  dissertation  (Gutzwiller  &  Clegg,  2012; 
Gugerty,  in  press;  Harbour  et  ah,  2012;  Jen-li  et  ah,  2013;  Jones  &  Endsley,  2012; 
Sulistyawati  et  ah,  2011;  Vidulich  &  Tsang,  2012).  Therefore,  this  work  tested  explicit 
hypotheses  regarding  specific  abilities  that  were  found  to  contribute  to  situation 
awareness  making  it  predictable.  The  results  of  this  study  fills  in  a  key  and  critical  gap  in 
the  TMSA,  and  in  so  doing  enables  both  theoretical  refinement  and  practical  applications 
such  as  improved  procedures,  training  for  pilots,  and  display  design  that  improves  flight 
safety,  this  problem  was  answered  and  the  solution  discovered. 
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Purpose 

The  primary  objective  of  this  quasi-experimental  quantitative  study  was  to  test  the 
predictive  value  of  these  particular  candidate  visuo-cognitive  abilities:  (a)  Visual 
Attentiveness,  consisting  of  Vigilance,  Focus,  and  Speed,  (b)  Visual  Perceptiveness, 
consisting  of  Prudence,  Consistency,  and  Stamina,  and  (c)  Visuospatial  Working 
Memory,  which  consisted  of  Working  Memory  that  is  stored  in  the  visuospatial 
sketchpad  in  the  mind,  as  predictors  of  situation  awareness,  e.g.  the  effects  of  visual 
ability  on  the  level  of  situation  awareness.  Therefore,  this  project  tested  the  predictive 
value  of  these  variables  as  factors  in  a  particular  task  to  be  performed  and  the  eventual 
outcome  of  situation  awareness.  As  Pavlov,  Watson,  and  Skinner  performed  field 
experiments  in  order  to  contribute  to  psychological  theory  (Cervone  &  Pervin,  2007),  a 
field  research  in  the  paradigm  of  neuroergonomics  (Parasuraman,  Christensen,  & 

Grafton,  2012)  employing  quasi-experimental  repeated-measures  (within-participants), 
was  conducted  for  this  study.  Further  inquiry  involving  visual  cues  clarified  the  role  of 
these  abilities  in  order  to  make  theory-driven  predictions  for  situation  awareness 
(Harbour  et  ah,  2012;  Vidulich  &  Tsang,  2012;  Wickens  &  McCarley,  2008).  Statistical 
analyses  of  the  data  indicated  that  this  purpose  was  achieved. 

Method 

The  primary  purpose  of  this  quasi-experimental  quantitative  study  was  to  test 
visual  perceptiveness,  attentiveness,  and  spatial  working  memory  as  predictors  of 
situation  awareness.  In  order  to  maximize  ecological  validity  and  minimize  nuisance 
variance  due  to  heterogeneous  participants,  this  project  focused  on  the  controlled 
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assessment  of  the  visuocognitive  abilities  of  Air  National  Guard  pilots,  with  aetual 
piloting  as  an  entirely  real  task  suitable  for  measuring  resulting  situation  awareness. 

This  was  the  most  practieal,  effective,  and  realistic  method  to  perform  the 
research  that  addresses  the  problem  and  the  purpose.  As  an  example  alternative 
approach,  a  completely  laboratory  based  study  utilizing  the  typical  convenience  sample 
of  college  undergraduates  could  be  accomplished;  however,  one  would  expect  significant 
individual  differences  as  well  as  serious  concerns  about  the  measurement  of  situation 
awareness  outside  any  meaningful  task  context.  If  this  work  is  wholly  successful,  it  will 
result  in  significant  elaboration  of  theoretical  models  of  situation  awareness  as  well  as 
enabling  and  focusing  efforts  to  improve  situation  awareness. 

In  order  to  induce  meaningful  variation  in  the  outcome  measures  of  situation 
awareness,  task  difficulty  in  the  flight  task  was  manipulated  between  two  levels,  an  easy 
or  low  difficulty  level  associated  with  efficient  and  easy-to-use  information  displays  and 
a  hard  or  high  difficulty  level  associated  with  ineffective  information  displays.  Each  pilot 
was  assessed  for  the  three  candidate  visual  abilities  (attentiveness,  perceptiveness,  and 
visuospatial  working  memory).  This  approach  resulted  in  a  quasi-experimental  design 
that  made  use  of  repeated  measures  comparisons  (Cozby,  2001;  see  Table  3)  four  times 
per  subject,  in  order  to  achieve  required  statistical  power.  The  experiment  was  organized 
in  much  the  same  manner  as  a  strict  experimental  design  however  due  the  operational 
realism  involved  it  lacked  complete  randomness  (Creswell,  2009;  de  Vaus,  2001;  Moore, 
2007).  This  lack  of  randomness  was  recognized,  and  was  accounted  for  during  the  entire 
experimental  process  (Trochim  &  Donnelly,  2008).  This  tradeoff  between  experimental 
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control  and  ecological  validity,  while  a  compromise,  should  improve  the  validity, 
generalizability,  and  signifieance  of  this  study  (Alasuutari,  Bickman,  &  Brannen,  2008). 

Correlation  and  causation  analyses  utilize  mathematical  tools  such  as  the  Analysis 
of  Variance  (ANOVA)  to  study  effects  on  one  dependent  variable  by  more  than  one 
independent  variable,  along  with  multiple  regression  analysis  and  the  Pearson  correlation 
(Grim  &  Yamold,  2006).  The  within-subjects  and  factors  repeated-measures  ANOVA, 
moreover  with  multiple  regression  analysis  were  used  to  measure  the  main  effeets  of  the 
independent  variables  (three  visual  abilities)  the  interaetions  among  the  independent 
variables,  the  importance  of  the  dependent  variable  (situation  awareness),  and  the 
strength  of  association  between  these  variables. 

Limitations 

Some  of  the  pre-existing  factors  such  as  the  subjeet  pilot’s  previous  experience 
using  HUDs  and  his  or  her  level  of  piloting  skills  were  not  taken  into  account.  In 
addition,  there  are  an  untold  number  of  possible  outside  influenees  that  may  affect  the 
results,  such  as  sleep  quality,  family/emotional  disturbanee,  diet,  ete.  (Cervone  &  Pervin, 
2007).  There  was  not  complete  randomness.  Even  with  these  disadvantages,  because 
the  shorteomings  were  recognized  throughout  this  study,  and  repeated  measures  were 
taken,  the  results  should  still  be  viable  and  valuable  (Creswell,  2009;  de  Vaus,  2001; 
Trochim  &  Donnelly,  2008).  The  quasi-experimental  design  was  the  best  ehoice  for  this 
study  and  was  very  useful  in  generating  results  for  trends  and  relationships  between 
variables  (Creswell,  2009;  de  Vaus,  2001;  Moore,  2007;  Troehim  &  Donnelly,  2008). 

The  experiment  proceeded  with  a  variable  or  variables  being  eompared  between  different 
IDs  used  (HDD  and  HUD)  and  within  group  over  a  period  of  time.  The  quasi- 
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experimental  design  allowed  for  statistical  analysis  to  take  place,  without  extensive  pre¬ 
screening  and  randomization  needed  to  be  undertaken,  reducing  the  time  and  resources 
required  for  experimentation  (as  this  study  is  a  dissertation  therefore  there  are  limitations 
to  funds  and  time  to  complete). 

Ethical  Dimensions 

The  subjects  flew  the  aircraft  with  an  associated  very  low  and  minimal  risk  level. 
Serious  thought  was  devoted  to  these  flights,  the  possible  risks  involved,  and  possible 
mitigations  for  those  risks  such  as  a  “knock  it  off’  call  by  anyone  of  the  subjects  or  the 
researcher.  The  study  sought  to  maximize  realism  in  order  to  develop  more  robust 
answers  and  theory — ^with  pilots  flying  the  actual  aircraft,  rather  than  using  simulators. 
The  risk  was  discovered  to  be  small  or  minimal  during  both  the  NCU  IRB  (see  Appendix 
B)  and  USAF  IRB  (see  Appendix  C).  This  study  was  conducted  with  an  AFRL  IRB  and 
NCU  IRB  approval;  participants  completed  comprehensive  written  informed  consent,  and 
all  appropriate  USAF  boards  were  performed  (see  Appendices’  B  and  C). 

The  rights,  privacy,  responsibilities,  and  safety  of  the  researcher  and  subjects  was 
strictly  respected  and  adhered  to  at  all  times  for  this  study  (Creswell,  2009). 

Implications:  Theoretical  and  Practical  Significance 

This  study  provides  significance  to  the  scientific  field  of  psychology  that  is  both 
theoretical  and  practical.  Theory  in  psychology  aims  to  explain  or  describe  at  some  level 
the  operation  or  process  of  the  brain  or  mind  from  either  or  a  combination  of  a 
performance,  an  emotion,  a  cognitive,  and  or  a  behavior  perspective  (Carlson,  2010; 
Cervone  &  Pervin,  2007;  Matlin,  2008).  The  term  theory  does  not  have  an  absolute 
single  one-size  fits  all  definition  (Benthem,  2007;  Flarlow,  2009).  Psychology  contains  a 
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range  of  differing  ideals  that  define  theory  and  a  broad  assortment  of  uses  of  the  term 
“theory”,  some  that  are  very  seientific  and  some  that  are  not  (Harlow,  2009). 

Theory 

Psyehology,  like  physics  is  the  study  of  a  vast  and  confusing  phenomena  to 
comprehend  and  explain,  the  human  mind  versus  the  cosmos  at  both  the  macro  and  micro 
level  contain  similarity.  Theory  in  cognitive  psychology  lends  itself  to  being 
mathematically  analyzable.  Cognitive  psychology  typically  is  human  performance  based. 
Viewing  theory  using  parsimonious  taxonomies  towards  differentiation,  evaluation,  and 
formulation,  enriches  the  perspective  of  theory,  therefore  providing  the  catalyst  to  create. 
In  the  view  of  three  scholars,  Lynham,  DiMaggio,  and  Heinen,  theory  can  be  envisaged 
in  many  different  ways.  There  exist  three  fundamental  views  of  what  constitutes  theory 
based  on  what  DiMaggio  (1995)  surmises  as,  “theory  as  covering  laws,  theory  as 
enlightenment,  and  theory  as  narrative”  (p.  391).  Citing  Habermas’s  three-perspective 
classification,  Lynham  (2002)  indicated  these  three,  partition  theory  as,  empirical- 
analytical,  interpretive,  and  critical  (p.  225).  Parsimoniously  categorizing  theories 
further,  according  to  Heinen  (1985)  in  reality  there  are  only  two  classes:  concatenated 
theory  and  hierarchical  theory  (pp.  417-418).  After,  much  review  this  learner  and 
candidate  believes  that  the  three  views  expressed  in  DiMaggio’s  (1995)  work  capture  and 
encompass  most  current  and  varying  scholarly  views  of  theory,  today,  and  for  the 
foreseeable  future.  Two  fundamental  aspects  will  be  covered  as  they  are  apropos  for  this 
study  and  the  TMSA. 

Theory  as  Covering  Laws.  This  is  the  positivist  view  of  theory,  e.g.  the  Hick- 
Hyman  Law.  Sutton  and  Staw  (1995)  considered  this  theory  as  being  narrow-minded  and 
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too  robotic.  Sutton  and  Staw  (1995)  discarded  the  ideas  that  theory  must  be  based  on;  a) 
rejeeting  the  null  hypothesis,  and  b)  only  items  in  the  world  that  ean  be  quantifiable  and 
measured  exaetly.  On  the  eontrary,  most  soeial  seientists  and  physieist  would  view 
theory  as  law  as  being  proper,  and  in  turn  view  Sutton  and  Staw  as  being  extremists 
(DiMaggio,  1995).  DiMaggio  (1995)  shared  the  same  opinion  with  Sutton  and  Staw 
(1995),  however.  This  anti -positivist  view  regards  theory  as  law  as  being  too  absorbed 
into  the  Pearson  produet  squared  {R^)  as  the  “judge”  for  good  theory  and  that  positivist 
focus  too  much  on  clarifying  variance  rather  than  regularities  (DiMaggio,  1995).  On  the 
other  hand,  Friedman  (1953)  viewed  the  (/?^)  notion  as  being  eorreet  and  that  theory  as 
law  is  truth,  therefore  providing  if  the  (/?^)  value  is  high  it  is  then  good  theory,  Sutton 
and  Staw  argue  it  only  provides  the  “what”,  not  the  “how”  or  “why”.  One  eould  take  the 
view  that  theory  as  law  to  be  mostly  deduetive  in  nature  by  using  data  to  test  theory 
(Eisenhardt  &  Graebner,  2007;  Goel  &  Dolan,  2004;  Sternberg,  2009).  Based  on 
Einstein’s  (1916)  Relative  theory  one  eould  infer  that  Einstein  would  view  good  theory  to 
be  theory  as  law  whereas.  Quantum  theorist  may  view  theory  of  enlightenment  as  good 
theory,  instead,  however,  this  eandidate  and  researeher  views  them  both  as  good  theory 
when  eombined.  The  TMSA  in  its  eurrent  state  is  not  theory  as  law  but  is  theory  as 
narrative. 

Theory  as  Narrative.  This  is  the  functionalist  view  of  theory.  DiMaggio  (1995), 
eonsiders  this  theory  as  being  a  mix  of  theory  as  law  and  theory  as  enlightenment,  theory 
as  narrative  eombines,  narrative  aeeounts  of  proeesses  along  with  importanee  on 
empirieal  tests  of  feasibility,  Collins  (1981)  has  an  austere  form  of  this  view  and  refers  to 
it  as  micro-translation  whereas  theory  does  eontain  a  hypothesis,  but  it  merely  speeifies 


160 


the  consistencies  among  variables  to  include  relationships,  while  being  simultaneously 
tied  in  with  plausible  accounts  observing  and  predicting  how  the  actual  interactions  and 
associations  are  produced.  The  approach  of  Farrao  (1989)  for  theory  is  similar  but  yet 
more  aggressive  in  that  a  baseline  generator  in  the  form  of  a  model  replicates  human 
behavior  based  on  a  set  of  governing  principles.  Through  computer  simulation,  utilizing 
this  model  observing  the  output  and  distributions  will  culminate  into  the  conclusions. 
DiMaggio,  Sutton  and  Straw  view,  this  view  of  theory  being  narrative  in  much  the  same 
way  (Dimaggio,  1995;  Sutton  &  Straw,  1985),  this  opinion  of  the  narrative  approach  in 
respect  to  theory  is  in-between  the  views  of  Fararo  and  Collins.  That  is,  theory  as 
narrative  is  a  more  equal  mix  of  theory  as  law  and  theory  as  enlightenment,  whereas 
Fararo  leans  heavier  towards  theory  as  law  and  Collins  leans  heavier  towards  theory  as 
enlightenment.  The  TMSA  in  its  current  state  is  theory  as  narrative. 

Scientific  Theory.  Rychlak  (1968)  as  described  in  Gelso  (2006),  and  Ellis  and 
Levy  (2009),  indicate  four  primary  functions  underpin  theory:  (a)  descriptive,  (b) 
delimiting,  (c)  generative,  and  (d)  integrative.  Narrative  answers  the  why,  delimiting 
bounds,  generative  stimulates  new  research  adding  to  the  body  of  knowledge,  and 
integrative,  pursues  to  make  what  appears  to  be  dissimilar  similar  giving  a  sound 
connected  view.  Three  types  of  theory  that  are  apropos  to  psychology  (Bachman  & 
Schutt ,  2007;  Gay  &  Weaver,  2011;  Locke,  2007):  (a)  hypothetico-deductive  theory 
(Rynes  &  Gephart,  2004),  (b)  inductive-synthesis  theory  (Wacker,  1998,  2008),  and  (c) 
critical  theory  (Torraco,  2002).  Hypothetico-deductive  will  be  expressed  here. 

Hypothetico-deductive  theory  is  very  apropos  to  this  study.  It  is  based  on  the 
quantitative  research  method  using  operational  constructs  and  variables  (Rynes  & 


161 


Gephart,  2004).  It  involves  realism,  utilizing  measured  data  for  hypothesis  testing.  This 
type  of  theory  yields  theoretical  models,  laws,  and  universal  truths. 

In  addition,  the  type  of  psychology  theory  is  very  much  dependent,  upon  the  topic 
the  research  is  examining,  which  then  also  influences  the  type  method,  quantitative  or 
qualitative  used  (Trochim  &  Donnelly,  2008).  A  theory  could  be  a  scientific  law  or 
related  laws  such  as  Newton’s  Laws  of  Motion  (Griffith,  2007)  which  is  based  on  the 
quantitative  method,  or  a  set  of  constructs  for  understanding  phenomena  such  as  the 
individual’s  basis  for  fear  of  crime  in  the  United  Kingdom  grounded  in  the  qualitative 
method  (Harlow,  2009). 

Quantitative  Method  (Hypothetico-Deductive).  This  research  method  depends 
far  less  on  observations,  interviews,  focus  groups,  and  case  studies  as  found  in  the 
qualitative  method  where  theory  could  evolve  and  immerge.  The  quantitative  method 
relies  far  more  on  the  formulation  of  a  hypothesis  at  the  onset  based  on  previous  research, 
next  an  experiment  or  series  of  experiments  is  or  are  accomplished,  followed  by  the 
statistical  analysis  of  the  data  collected  to  test  the  hypothesis  in  order  to  answer  the 
research  questions.  For  theory  development  from  the  quantitative  method,  the  foremost 
elements  in  research  are  the  hypothesis,  what  is  varied  (independent  variable/s)  along 
with  what  is  measured  (the  dependent  variable/s)  in  order  to  test  the  hypothesis  (Harbour, 
2006;  Harbour,  2007;  Harbour,  2011;  Moore,  2007).  This  dissertation  employed  this 
method. 

The  literature  supports  the  ideology  that  theory  is  the  return  on  investment  or 
payoff  from  investing  in  or  the  performing  of  scholarly  research  (Gelso,  2006);  in  the 
case  of  the  author’s  dissertation,  it  would  be  theory  in  psychology,  cognitive 
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neuroscience,  engineering,  aviation,  and  human  factors,  combined.  The  Theoretical 
Model  of  Situation  Awareness  (TMSA)  (Endsley,  1995b)  is  theory  as  narrative  for  it 
combines  narrative  accounts  of  processes  along  with  emphasis  on  empirical  tests  of 
feasibility.  Endsley’ s  framework  is  grounded  in  hierarchical  levels  of  SA  and  is  based  on 
information  processing.  Eevel  1  SA  contains  the  perception  and  processing  of  cues.  An 
example  could  be  perceiving  the  environment  and  a  display  yielding  spatial  awareness 
(Endsley,  2012;  Wickens,  2007;  Wickens  et  ah,  2008).  Eevel  2  SA  is  the  comprehension 
of  the  current  situation  by  utilizing  the  information  gained  from  Eevel  1  SA  perceptions 
combined  with  individual  background  knowledge,  thereby  creating  a  situation  model 
(Endsley,  2000a,  1995a;  Wickens,  2007).  Eevel  3  SA  is  the  utilization  of  the  situation 
model  to  project  and  predict  the  future  state  (Endsley,  1995a,  2000b;  Sulistyawati, 
Wickens,  &  Chui,  2011;  Wickens,  2007).  Endsley’s  (1995a;  2002a)  view  of  SA  is  often 
utilized  as  the  basis  for  research  in  areas,  including  system  displays  and  military 
operations  (Eid,  Johnsen,  &  Brun,  2004;  Sulistyawati,  Wickens,  &  Chui,  2011).  SA  is 
difficult  and  challenging  to  quantify,  it  is  viewed  as  a  complicated  tangible,  that  exists 
and  necessitates  an  acute  sense  for  visual  signals  (Endsley,  2012;  Dekker  &  Hollnagel, 
2004;  Elliott,  et  ah,  2009;  Jodlowski,  2008;  Parasuraman,  Sheridan,  &  Wickens,  2008) 
feeding  on  the  perception  of  the  elements  in  one’s  world  within  the  dimensions  of  space 
and  time  (Endsley,  2012).  However,  in  argument  there  were  at  least  34  different  and  or 
varying  definitions  of  SA  with  little  consensus  in  the  scientific  community  (Cass,  2011; 
Dekker  &  Hollnagel,  2004;  Jodlowski,  2008;  Rousseau,  et  ah,  2004;  Wickens,  2008)  and 
the  same  existed  for  the  exact  composition  of  SA  to  include  its  predictors  (Blandford  & 
Wong,  2004;  Elliott,  et.  ah,  2009;  Gorman,  et  ah,  2006;  Jodlowski,  2008;  Rousseau,  et. 
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al,  2004).  To  complicate  matters  further,  there  are  at  least  three  different  theories  for  SA 
(Stanton,  Chambers,  &  Piggott,  2001),  the  TMSA  -  a  three  level  model  that  is  a  cognitive 
theory  that  uses  an  information  processing  approaeh  (Endsley,  1995a),  the  theory  of 
activity  model  to  describe  SA  (Bedny  &  Meister,  1999),  and  the  perceptual  cycle  theory 
model  that  is  an  eeological  approaeh  (Niesser,  1976;  Smith  &  Hancock,  1995),  these 
theories  diverge  in  their  foundational  psychologieal  methodology  (Salmon,  et  ah,  2008). 

This  fuels  the  debate  on  how  to  best  measure  SA  even  though  at  hand  are  a 
plethora  of  approaches  to  assessing  SA  and  the  copious  theoretical  debate  that  hinges 
over  whether  SA  refers  to  the  process  of  gaining  awareness,  or  the  product  of  it,  or  a 
eombination  of  the  two  (Salmon,  et  al.  2008).  The  eontinuing  debates  over  SA  illustrate 
the  need  to  refine  the  theory  (Dekker  &  Hollnagel,  2004;  Patrick  &  Morgan,  2009; 
Wickens,  2008).  Consequently,  there  simply  was  no  established  theory  as  law  of  SA,  and 
therefore,  no  specific  mathematical  equation  or  variable  relationships  that  include  SA 
predictors.  The  TMSA  utilizes  an  information  processing  approach,  not  an  activity 
approach,  nor  an  ecological  approach  (Stanton,  et  ah,  2001)  it  is  a  functional  model  for 
assessing  distinet  levels  of  insight  in  a  realistic  fashion.  Further  controversy  existed 
within  the  TMSA,  Endsley’ s  theory  emphasizes  pereeption  and  eomprehension  of  the 
environment  amid  projeetion  into  the  future  however  it  did  not  contain  enough 
granularity  or  accuracy  in  the  area  of  human  pereeption  (Elliott,  et.  ah,  2009;  Gorman,  et 
al,  2006;  Jodlowski,  2008;  Stanton,  et  ah,  2001;  Wickens,  2008).  The  TMSA  did  not 
emphasize  refleetive  relationships  between  mental  models  and  knowledge  of  the  present 
system.  SA  may  be  a  unique  psychologieal  construct  on  its  own  (Bell  &  Eyon,  2000; 
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SA  is  a  vital  paradigm  that  continues  to  provoke  controversy  (Wickens,  2008). 
What  is  needed,  but  missing  from  Endsley’s  theory  of  SA  (2012)  is  an  approach  for 
predicting  the  pilot’s  ability,  to  perceive  the  display  cues  while  using  the  display  in  order 
to  operate  an  aircraft  or  UAV  (multi-tasking),  predicting  Level  1  SA  remained  a  needed 
ehallenge  (Harbour,  et  al.  2012;  Wickens  &  MeCarley,  2008).  Finally,  as  any  signifieant 
concept  should,  SA  has  spawned  some  degree  of  rigorous  academic  debate  (Dekker  & 
Hollnagel,  2004;  Dekker  &  Woods,  2002;  Patrick  &  Morgan,  2009;  Wickens,  2008). 

An  understanding  of  the  visual  proeessing  faetor  was  incomplete  and  research  to 
refine  measurement  of  these  abilities  needed  to  be  accomplished.  Elliot,  Wiekens, 
Parasuraman,  Christensen,  and  many  others  have  been  performing  research  to  solve  this 
unknown  (Endsley,  2012;  Vidulieh  &  Tsang,  2012).  While  filling  in  a  gap  in  the  TMSA 
specifieally  at  Level  I,  this  dissertation  study  provides  Theory  as  Law  to  the  existing 
Theory  as  Narrative  TMSA  with  the  following  mathematical  equation: 

Predicted  ZsA  =  11.646  —  0.606Zyp  —  O.OSlZya  —  0.334Zysvi;m- 
This  researeh  was  successful  by  providing  continuity  to  the  breach  in  the  TMSA  and 
offers  an  Enhanced-TMSA  by  providing  the  specific  measurable  neurocognitive  attributes 
that  are  required  to  feed  Level  1  SA  -  perception,  allowing  SA  to  be  quantifiably 
predictable.  This  is  a  psychological  theory  that  could  be  appropriate  in  many  different 
domains  involving  human  behavior. 

There  is  a  relationship  between  the  results  of  this  dissertation  and  current  research 
in  computational  intelligence,  preeisely  the  QUEST  approaeh  (Rogers,  2009).  The  main 
objective  of  QUEST  is  to  develop  a  general-purpose  eomputational  intelligence  system 
that  captures  the  advantageous  engineering  aspects  of  qualia  based  solutions.  Ultimately, 
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a  QUEST  system  should  have  the  ability  to  deteet,  extrieate,  and  portray  entities  in  the 
environment,  to  inelude  a  representation  of  its  self  and  possess  self-awareness.  In  a 
sense,  then,  QUEST  is  working  towards  a  Theory  of  Conseiousness  Awareness  (Rogers, 
2014).  In  so  doing,  QUEST  is  utilizing  an  emerging  theory  in  psychology  referred  to  as 
Dual-process  or  Dual-system  theory  (Evans  &  Stanovich,  2013).  Dual-process  theory  is 
premised  on  the  idea  that  human  behavior  and  decision-making  involves  autonomous 
processes  (Type  1)  that  produce  default  responses  involving  an  implicit  process  unless 
interceded  upon  by  distinctive  higher  order  reasoning  processes  (Type  2).  Type  2,  on  the 
other  hand  involves  an  explicit  process  and  burdens  working  memory.  Typical  correlates 
associated  with  Type  1  are  automatic,  non-consciousness,  and  basic.  Type  2  is  typically 
associated  with:  controlled,  consciousness,  and  complex  (see  Eigure  14).  With  the 
uncontrolled  nature  of  in-flight  events,  one  may  assume  that  pilots  had  to  engage  both 
types  of  processing  on  any  given  flight.  The  neurocognitive  predictors  may  reflect 
efficiencies  in  Type  1  processing  that  translate  to  a  reduced  need  for  Type  2  and  attendant 
lower  workload  and  higher  SA.  Stated  differently,  pilots  with  stronger  perceptual  and 
attentive  capabilities  may  need  to  engage  the  effortful  Type  2  system  less,  thus  preserving 
spare  capacity  for  maintaining  SA. 


166 


Figure  14.  Dual  Processing  Theory  involving  both  Type  1  and  Type  2  proeesses  (Evans 
&  Stanovich,  2013). 


During  the  24  flights,  there  were  unexpeeted  queries  (UQ)  encountered  by  the 
pilot  as  well  as  expected  queries  (EQ)  based  on  environmental  stimuli.  During  the  sorties 
both  UQ  and  EQ  were  occurring.  However,  sinee  it  was  not  central  to  this  dissertation’s 
problem  statement  and  purpose  it  was  not  ferreted  out  as  to  whether  or  not  any 
differences  in  UQ  and  EQ  for  situation  awareness  and  cognitive  workload  outcome 
existed.  Consequently,  in  working  towards  a  Theory  of  Consciousness  Awareness  this 
will  need  to  oeeur  in  future  research  based  on  environmental  stimuli  (Rogers,  2014). 
Practical  Application 

The  most  useful  psychologieal  theory  is  one  that  is  systematie,  testable,  and 
comprehensive  with  the  ultimate  goal  of  being  able  to  construct  the  theory  into  one  of  the 
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tools  to  be  applied  by  the  psyehologist  or  engineer  in  praetical  applieation  to  better 
understand,  improve,  and  predict  human  thought  and  behavior  (Santrock,  2008).  Basic 
research  expands  science  and  or  creates  new  theory,  and  is  a  goal  of  the  Ph.D. 
dissertation.  The  product  of  which  is  a  significant  advancement  transversely  on  a  broad 
front  of  knowledge  expansion  of  natural  and  social  phenomena  (Jain,  Triandis,  &  Weick, 
2010).  To  obtain  a  deepened  comprehension  of  the  subject  matter  under  study,  in  order 
to  increase  knowledge  and  theory  in  the  science  or  field  without  necessarily  a  particular 
application  in  mind,  is  the  main  object  of  basic  research  (Jackson,  2009;  Jain,  et  ah, 
2010).  On  the  other  hand,  applied  research  uses  the  very  principles  that  basic  research 
discovers  towards  practical  application  (Jackson,  2009).  A  practical  implementation  is 
focused  on  finding  the  means  by  which  a  specific,  recognized  need  may  or  can  be  met 
(Jain,  et  ah,  2010).  Basic  research  discovers  the  theories  in  science  and  potentially  finds 
what  theory  maybe  better  suited  than  another  to  solve  a  problem.  Whereas  the  results  of 
applied  research  which  could  be  a  practical  application  using  that  discovered  theory  in 
science  and  applying  it  to  a  specific  issue  in  order  to  make,  and  or  build,  and  or  apply  it 
for  something  new  to  full-fill  a  need,  treatment  plan,  or  safety.  Basic  research,  applied 
research,  and  practical  application  are  equally  important  because  many  treatments  and 
procedures  that  have  been  developed  to  help  humans  result  from  research  and 
implementation  (Jackson,  2009).  The  Enhanced  -  TMSA  can  be  applied  by  utilizing  the 
SEEV  Model  improving  SA,  and  or  employing  cognitive  training  to  enhance  these 
individual  neurocognitive  factors  resulting  in  improved  flight  safety. 


How  the  Enhanced  -  TMSA  Can  Be  Applied.  Applying  the  Enhanced  -  TMSA 
to  cockpit  design,  pilot  training,  and  pilot  selection  could  offer-up  the  quickest  benefits 
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today.  The  practical  need  is  that  approach  and  landing  accidents  are  of  concern, 
especially  at  austere  locations  or  where  the  weather  creates  poor  visibility  (Arthur, 

Prinzel,  Williams,  &  Kramer,  2006;  Bulkley  et.  al,  2009).  From  1980  to  2007,  in 
commercial  aviation  983  accidents  occurred,  involving  multiengine  jet  aircraft  that 
weighed  12,500  pounds  or  more  with  glass  cockpits,  and  were  due  to  causal  factors  such 
as  reduced  SA  and  increased  mental  WL  of  the  pilot,  combined  with  limited  visibility, 
and  operations  into  ascetic  places  (Flight  Safety  Foundation,  2009).  Deficient  situation 
awareness  can  lead  to  fatal  accidents,  in  particular  the  number-one  killer  in  commercial 
aviation,  CFIT  (Wickens  &  McCarley,  2008).  It  has  been  reported  that  if  situation 
awareness  could  be  improved  this  accident  potential  could  be  reduced  by  33  to  73 
percent,  saving  lives  and  an  estimated  $1  million  per  commercial  and  military  aircraft 
over  the  next  10  years  (FSF,  2009;  Prinzel  &  Risser,  2004;  Rockwell  Collins,  2000). 

Based  upon  leading  safety  experts’  evaluations  as  derived  from  information 
obtained  in  flight  simulators,  it  is  believed  that  those  accidents  could  have  been  prevented 
had  an  appropriately  HUD  been  installed  in  the  cockpit  and  SA  been  improved  (Flight 
Safety  Foundation,  2009;  Kim,  2009)  and  this  dissertation  supports  those  evaluations. 

For  the  practical  application  designer  the  Enhanced-TMSA  would  enable  optimization  by 
making  valuable  information  more  salient,  and  it  can  reduce  the  effort  of  switching 
between  sources  with  high  bandwidth  and  likely  PRPs.  Installing  a  HUD  in  every 
aircraft  is  the  quickest  way  to  achieve  this. 

With  the  application  of  the  Enhanced-TMSA,  aircraft  cockpits  could  be  designed 
to  where  the  location  and  appearance  of  symbology  on  the  display  more  efficiently 
correspond  in  both  appearance  and  place  with  the  actual  environment  in  the  most 
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efficient  prioritized  manner  (FSF,  2009).  Synthetic  and  enhanced  vision  HUDs  could  be 
designed  and  installed  by  using  the  Enhanced-  TMSA,  and  SEEV  application  derived 
from  MRT  (Bulkley  et.  al,  2009).  Additionally,  these  could  be  used  in  NextGen  airspace 
for  the  planned  control  of  both  Unmanned  Arial  Vehicles  (UAVs)  and  manned  aircraft 
flying  in  the  same  airspace  simultaneously  within  close  proximity  (Gore,  et.  al,  2009).  In 
automobiles,  the  applications  could  be  for  3D  maps  (Wickens  &  McCarthy,  2008; 
Cannon,  Nguyen,  &  Regli,  2012)  improving  safety.  These  applications  could  be  soundly 
grounded  in  the  Enhanced-  TMSA,  based  on  the  solid  psychological  theory  presented  here 
and  reliable  engineering  applications  (Sarter,  2012).  Consequently,  the  reasoning  linking 
implementation  and  theory  is  sound,  the  particular  links  being  Selective  attention,  Task 
selection,  Allocation,  Demand,  and  Multiplicity  (Wickens  &  McCarthy,  2008). 
Consequently,  moreover  computerized  cognitive  training  (CCT)  should  be  considered  in 
order  to  potentially  improve  these  abilities  potentially,  as  well. 

The  Study’s  Impact  on  the  Literature 

There  was  a  lack  and  need  for  additional  psychological  theory  that  is  based  on  the 
neurocognitive  abilities  to  perceive  the  visual  display’s  cues,  predicting  situation 
awareness  and  workload  (Harbour,  et.  ah,  2012;  Kim,  2009;  Endsley,  2012;  Sulistyawati, 
et.  al.,  2011;  Vidulich  &  Tsang,  2012)  succinctly  quantitatively  grounding  the  TMSA. 
This  was  covered  and  expanded  upon  further  in  the  Eiterature  Review.  This  Ph.D.  study 
has  made  a  solid  and  expansive  contribution  to  the  literature  in  behavioral  science  by 
addressing  a  fundamental  gap  in  cognitive  situation  awareness  theory  by  way  of 
examining  both  pilot  neurocognitive  factors  and  flight  display  three-dimensional 
placement,  and  how  and  why  they  may  well  affect  situation  awareness  and  workload. 
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Flying  modern  military  aircraft  and  unmanned  aerial  vehicles  is  indeed  operating 
a  complex  system,  performing  several  tasks  at  the  same  time  that  increase  the  pilots’ 
mental  WL.  Accurate  mental  workload  assessment  and  SA  prediction  is  critical  because 
WL  significantly  affects  human  performance,  and  a  better  understanding  of  the  available 
mental  capacity  (for  SA)  and  WL  relationship  is  vital  (Kang,  2008).  For  instance,  up  to 
82%  of  highway  accidents  result  from  distraction  (inattention),  and  today’s  workforce  see 
the  car  as  a  mobile  office  (the  birth  of  the  “multi-tasking”  next  generation)  (Wickens  & 
McCarley,  2008). 

Nineteen  quantitative  experiments  and  fifteen  research  articles  have  illustrated  the 
need  and  attempted  to  discover  the  detailed  links  between  vision,  cognition, 
predictability,  and  situation  awareness.  Over  this  15-year  period,  up  to  and  including 
current  present  day,  the  research  has  gotten  closer,  illustrating  the  continuing  need  to  fill 
this  gap  in  the  TMSA.  Additionally,  three  other  theories  and  or  models  have  links  to  the 
TMSA  and  they  are  Applied  Attention  Theory  (AAT)  (Wickens  &  McCarley,  2008), 
Multiple  Resource  Theory  (MRT)  (Sarter,  2012),  and  the  Salience  Effort  Expectancy 
Value  (SEEV)  Model  (Wickens  &  McCarley,  2008).  Eive  quantitative  experiments  and 
six  research  articles  have  shown  these  to  be  linked  and  influenced  by  the  TMSA; 
consequently,  this  research  benefits  those  theories  as  well.  This  dissertation  identified 
critical  variables  underlying  the  formation  of  situation  awareness  as  well  as  the 
relationships  among  these  variables  filling  this  scientific  void  in  theory  (Elliott  et  al, 
2009),  grounding  the  TMSA  in  concrete  and  quantifiable  perception  and  cognitive 


processes. 
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What  was  known  is  that  situation  awareness  plays  a  vital  role  in  dynamic 
decision-making  environments  (St.  John  &  Smallman,  2008).  However,  further 
controversy  existed  regarding  specific  details  of  TMSA.  Endsley’s  theory  emphasizes 
perception  and  comprehension  of  the  environment  amid  projection  into  the  future; 
however,  it  did  not  contain  enough  granularity  or  accuracy  in  the  area  of  human 
perception  (Elliott  et.  ah,  2009;  Gorman  et  ah,  2006;  Jodlowski,  2008;  Stanton  et  al, 
2001;  Wickens,  2008).  The  TMSA  did  not  emphasize  reflective  relationships  between 
mental  models  and  knowledge  of  the  present  system. 

Endsley’s  view  of  situation  awareness  the  TMSA  is  often  utilized  as  the  basis  for 
research  in  areas  such  as  system  displays  and  military  operations  (Sulistyawati,  Wickens, 
&  Chui,  2011).  Eevel  3  situation  awareness  is  the  utilization  of  the  situation  model  to 
project  and  predict  the  future  state  (Sulistyawati,  Wickens,  &  Chui,  2011).  What  was 
needed  in  this  research  was  to  specifically  investigate  neurocognitive  characteristics  such 
as  visual  attention  and  visual-spatial  working  memory,  and  the  effects  on  pilot  workload 
and  situation  awareness  (Sulistyawati,  Wickens,  &  Chui,  2011).  There  was  a  lack  and 
need  for  additional  psychological  theory  that  is  based  on  the  neurocognitive  abilities  to 
perceive  the  visual  display’s  cues,  predicting  situation  awareness  and  workload  that  is 
Eevel  1  situation  awareness  (Sulistyawati,  Wickens,  &  Chui,  2011).  Research  continued 
to  struggle  to  find  detailed  links  between  the  cognitive  demands  on  pilots  and  situation 
awareness,  the  TMSA  was  and  is  vital. 

The  research  performed  by  Jen-li,  Ruey-Yun,  and  Ching-Jung  (2013)  examined, 
display  design  for  unmanned  aerial  vehicle  (UAV)  monitoring,  and  its  effects  on  operator 
situation  awareness,  performance,  and  mental  workload  involving  the  TMSA.  The 
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operator  in  UAV  flights  has  to  rely  primarily  on  vision  in  this  agent-based  system, 
consequently  the  question  remains  to  be  what  role  does  display  design  and  human  visual 
abilities  play  in  the  human-robot  interface  in  order  to  enhance  situation  awareness  (Jones, 
Connors,  &  Endsley,  2011),  again  illustrating  the  need  to  expand  the  TMSA. 

The  results  illustrated  that  compared  to  the  conventional  display,  the  effects  of 
situation-augmented  display  on  flight  completion  time  and  abnormality  detection  time 
were  robust  across  different  workloads  but  error  rate  and  perceived  mental  workload 
were  unaffected  by  the  display  type.  With  the  increasing  complexity  of  new  automation 
technology,  visual  processes  to  aid  the  control  operator’s  situation  awareness  remain  a 
significant  challenge  for  the  field.  An  important  point  gained  from  this  study  is  that  the 
Level  1  SA  from  the  TMSA  still  presented  problems  for  these  researchers  in  that  they  had 
difficulty  in  measuring  the  effects  between  operator  visual  abilities  and  display  usability 
(Jen-li  et  ah,  2013).  More  studies  were  needed  to  address  these  unresolved  and 
important  issues  (Jen-li  et  ah,  2013)  expanding  and  enhancing  the  TMSA. 

The  very  recent  in-press  work  by  Gugerty,  investigated  and  discussed  probable 
component  processes,  both  perceptual  and  cognitive,  that  make  up  the  ability  for 
situation  awareness  during  real-time  task.  The  psychological  field  needs  a  better 
understanding  of  the  foundation  of  theoretical  models,  so  that  through  empirical 
evidence  a  better  conception  of  situation  awareness  and  its  component  processes  can  be 
achieved  (Gugerty,  in  press).  Situation  awareness  involves  elements  such  as  the 
processes  of  focal  vision,  including  attention  as  well  as  ambient  vision  processes, 
including  attention  capture  by  abrupt  peripheral  events.  Situation  awareness  is  a 
complex  process  that  requires  further  assessment  (Gugerty,  in  press). 
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Gugerty  posited  that  the  eognitive  proeess  for  Level  1  SA  eould  potentially  be 
automatic,  and  therefore  would  place  almost  no  demands  on  cognitive  resources;  however, 
this  does  not  explain  the  cognitive  demand  of  attention  and  vigilance,  followed  by 
prudence  and  recognition.  Gugerty  contested  that  the  TMSA  three-level  view  of  SA 
processes  were  at  odds  when  it  comes  to  maintaining  SA  versus  acquiring  SA.  Future 
research  needs  to  narrow  or  broaden  this  view  (Gugerty,  in  press). 

The  work  of  Gugerty  indicated  that  increasing  SA  knowledge  further,  needs  to  be 
accomplished,  by  discovering,  measuring,  and  quantitatively  explaining,  describing,  and 
linking  these  particular  key  perceptual  factors  (visual  abilities),  with  the  goal  being  to 
objectively  fill  the  foundational  gaps  solidly  grounding  the  TMSA.  The  TMSA  is  linked 
to  the  AAT,  MRT,  and  the  SEEV  Model.  Wickens  and  McCarley’s  (2008)  Applied 
Attention  Theory  (AAT),  offered  further  support  for  this  research  in  that  AAT  indicates 
that  visual  attention  control,  scanning,  information  sampling,  visual  search,  spatial 
attention  and  displays  play  a  role  in  pilot  mental  workload,  which  in  turn  would  imply  an 
influence  on  situation  awareness  as  well,  linking  it  to  the  TMSA.  Additionally,  further 
AAT  clearly  illustrates  that  an  understanding  of  visual  processing  factors  operating  in 
dynamic  environments  related  to  attention  is  incomplete,  providing  further  evidence  of 
the  necessity  of  this  study  (Wickens  &  McCarley,  2008).  Moreover,  Multiple  Resource 
Theory  (MRT)  (Bulkley  et.  al,  2009;  Eei  &  Roetting,  2011;  Pickel  &  Staller,  2012; 

Sarter,  2012;  Vidulich  &  Tsang,  2012)  and  the  Salience  Effort  Expectancy  Value  (SEEV) 
model,  which  is  linked  to  MRT  through  the  visual  modality,  articulates  the  theoretical 
foundations  of  this  study. 
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One  assumption  of  the  TMSA  was  that  visual  and  cognitive  factors  influence 
situation  awareness  (Endsley,  1995a,  2012;  Vidulich  &  Tsang,  2012).  Previous  research 
had  indicated  that  complex  tasks  or  difficulties  in  operation  (visual  factors  and  task 
difficulty)  results  in  decreased  situation  awareness  (FAA,  2011;  Flight  Safety 
Foundation,  2009;  Kang,  2008;  Kang,  Yuan,  Fiu,  &  Fiu,  2008;  NTSB,  2009;  Wickens, 
2008).  In  this  study,  the  visual  factors  were  called  “visual  abilities”  and  were  a  function 
of  the  different  individuals  in  the  naturally  occurring  sample.  In  addition,  another 
cognitive  factor  was  called  “task  difficulty,”  was  manipulated  by  the  location  of  the 
display.  According  to  the  TMSA,  it  could  be  postulated  that  increased  visual  stress  and 
or  an  increase  in  mental  workload  will  decrease  the  level  of  situation  awareness, 
however,  the  effects  of  visual  ability  on  the  degree  of  situation  awareness  needed  to  be 
discovered. 

Existing  theoretical  models  explain  many  aspects  of  situation  awareness; 
however,  the  development  of  situation  awareness  out  of  basic  perceptual  abilities  was 
largely  unexplored.  This  quantitative  research  examined  basic  neurocognitive  factors 
including  visual  skills  and  working  memory  in  order  to  identify  their  specific 
contributions  to  the  formation  of  cognitive  situation  awareness,  to  in  turn  address  this  gap 
in  situation  awareness  (SA)  theory  that  must  be  linked  before  progress  could  be  made. 
Aircraft  piloting  was  used  as  a  task  where  situation  awareness  is  critical;  trained  USAF 
pilots  represent  a  relatively  homogenous,  already  expert  population  that  reduced  the 
nuisance  variance.  This  study  assessed  the  predictive  value  of  visual  attentiveness  (Va), 
perceptiveness  (Vp),  and  spatial  working  memory  (Vswm)  as  predictors  of  situation 
awareness  in  flight  under  varying  task  difficulty  using  repeated-measures  comparisons. 
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At  the  completion  of  this  field  experiment  the  data  was  analyzed,  and  the  tests  were 
statistically  significant  for  the  three  predictor  visual  abilities  Vp,  Va,  and  Vswm  as  task 
difficulty  was  varied.  In  addition,  multiple  regression  analyses  revealed  that  the  visual 
abilities  together  predicted  a  majority  of  the  variance  in  situation  awareness.  Moreover, 
the  Pearson  correlation  results  indicated  that  Vp  had  the  strongest  relationship  of  the  three 
neurocognitive  factors  for  the  overall  flight.  This  reveals  that  possessing  the  ability  to 
have  a  perceptivity,  to  be  insightful,  and  to  have  discernment  -  to  perceive  that  which  is 
ambiguous,  is  most  important.  This  appears  indicative  of  a  fused  explicit  and  implicit 
process.  Interestingly  during  high  task  difficulty  Va  had  the  strongest  correlation  with 
SA,  while  during  low  task  difficulty  Vswm  had  the  highest  correlation;  this  suggests  that 
under  high  demand  possessing  the  ability  to  concentrate  and  be  devoted,  to  be  detailed, 
and  responsive  becomes  the  largest  determinant.  While  under  low  task  demand 
responsiveness  and  insightfulness  is  less  essential  exposing  the  ability  to  have  working 
memory,  that  is  responsible  for  the  spatial  orientation  of  one  self  and  the  environment, 
allowing  it  to  become  the  largest  determinant  of  SA.  There  are  two  parts  to  working 
memory  (WM)  one  that  is  performing  passive  short-term  maintenance  of  what  is  going 
on  in  a  somewhat  static  environment,  just  using  the  visuospatial  sketchpad  and 
phonological  loop,  the  other  part  additionally  involves  active  manipulation,  such  as  the 
transformation  of  mental  representations  in  a  dynamic  environment  that  requires  central 
executive  involvement  (Kawasaki,  Kitajo,  &  Yamaguchi,  2010). 

Prior  research  was  incomplete  with  regards  to  the  exact  role  of  visual  processing 
abilities;  this  dissertation  research  refined  measurement  of  these  abilities.  The  objective 
measures  of  situation  awareness  and  workload,  in  actual  flight,  did  provide  the  basis  for 
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this  deeper  understanding  of  situation  awareness,  as  did,  testing  for  participants  levels  of 
visual  abilities  beforehand.  Given  these  findings,  the  most  important  predictive 
measures  for  future  studies  would  be  dynamic  visual  attention  and  spatial  tests.  For  the 
past  decade,  researchers  have  been  attempting  to  unravel  these  important  mysteries.  This 
work  results  in  a  significant  addition  and  expansion  of  the  existing  theoretical  model  of 
situation  awareness.  Through  inquiry  and  discovery  this  dissertation  identified  critical 
variables  underlying  the  formation  of  situation  awareness  as  well  as  the  relationships 
among  these  variables  filling  this  scientific  void  in  theory,  grounding  the  TMSA  in 
concrete  and  quantifiable  perception  and  cognitive  processes.  There  are  practical 
implications  as  well,  as  this  study  highlights  the  potential  for  improved  cockpit  design 
and  enhancing  training  by  targeting  attentional,  perceptual,  and  visuospatial  working 
memory  skill  learning. 

Jones  and  Endsley  (1996)  found  that  the  vast  majority  (77%)  of  human  errors  in 
aviation  pointing  to  situation  awareness  are  caused  by  difficulties  with  the  perception  of 
needed  information,  which  is  the  formation  of  Level  1  situation  awareness  (Figure  15). 

In  the  flying  environs,  pilots  must  make  time  critical  decisions;  therefore  efficient 
information  processing  becomes  paramount  and  the  ID  should  be  designed  so  that  the 
pilot  can  easily  perceive  the  PFl  in  order  to  aviate  safely. 


177 


Level  3:  Failure  to 
project  into  the  future 


needed  information 
TTA, 

Figure  15.  Errors  in  aviation  attributable  to  situation  awareness  (Jones  &  Endsley,  1996). 
A  significant  majority  of  errors  are  associated  with  perception,  which  for  aviation  is 
primarily  visual. 

A  reduction  in  attention  due  to  distractions  or  increased  effort,  which  can  result  in 
increased  workload,  has  been  found  to  undercut  situation  awareness  especially  in  flying 
tasks  and  poses  one  of  the  most  signiflcant  challenges  to  maintaining  situation  awareness 
(Endsley,  2012).  Evidence  has  been  found  that  individuals  with  better  situation 
awareness  seem  to  achieve  higher  scores  in  working  memory,  visual  processing,  temporal 
processing,  and  a  timesharing  ability  (Endsley,  2012;  Gugerty  &  Tirre,  1997).  This 
study  certainly  supports  this  premise. 
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Figure  16.  Level- 1  SA  expanded  in  Theoretieal  Model  of  situation  awareness  (adapted 
from  Endsley,  1995b).  This  level  of  situation  awareness  is  depieted  with  respeet  to 
individual  eapabilities.  The  study  addressed  the  potential  theoretieal  links  between  visual 
abilities,  and  situation  awareness. 


This  study  was  sueeessful  in  quenehing  the  diseontinuity  in  the  TMSA  and 
provides  an  Enhanced-TMSA  by  diseovering  the  speeifie  measurable  neuroeognitive 
attributes  (Va,  Vp,  Vswm)  that  are  the  visual  abilities  that  are  required  to  feed  Level  1  SA  - 
pereeption,  in  order  for  human  SA  to  be  predictable  (see  Figure  16). 

Recommendations 

The  findings  of  this  study  may  be  advantageous  to  psychological  researchers  by 
providing  further  insight  and  understanding  of  neuroeognitive  processes  that  are 
predictors  of  situation  awareness.  In  addition,  avionics  engineers,  cognitive  engineers, 
crew  system  engineers,  electronic  engineers,  and  aviation  psychologists  and  human 
factors  specialist  may  consider  the  findings  necessary  for  pilot  vehicle  integration  and 
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systems  engineering  proeesses  in  aircraft  and  UAV  design  and  development,  along  with 
discovering  new  ways  in  reducing  aircraft  accidents.  The  results  of  the  study  may  also  be 
a  base  for  future  investigations  of  neurocognitive  predictors  in  other  populations.  It  is 
recommended  from  this  study  that  further  research  be  extended  to  other  DoD  and  FAA 
aviation  populations. 

Replication  of  the  results  would  be  beneficial  in  validating  the  IVA  plus  and  N- 
back  as  a  practical  tool  to  provide  measurements  that  would  be  predictive  of  situation 
awareness  for  all  types  of  pilots,  automobile  drivers,  cyber  warfare  personnel,  surgical 
staff,  and  the  like.  Additionally,  a  replication  of  the  study  would  be  useful  to  other 
researchers  to  validate  and  determine  the  reliability  of  the  IVA  plus  and  N-back  to 
corroborate  the  statistical  findings.  Additionally,  Computerized  Cognitive  Training 
(CCT)  could  be  studied  to  see  if  it  improves  an  individual’s  visual  abilities. 

Further  research  should  also  be  accomplished  with  respect  to  (Va,  Vp,  Vswm), 
Qualia,  “Consciousness  Awareness”  versus  “Situation  Awareness”  archetypes,  and  Dual 
Processing  Theory,  in  pursuit  of  a  Theoretical  Model  of  Consciousness  Awareness 
(TMCA).  As  part  of  the  construction  of  the  TMCA,  there  is  a  need  for  a  Theory  for  the 
Type  2  processes  in  DPT  that  culminate  in  consciousness,  as  well  as  differences  in 
Awareness  Type  1  versus  Awareness  Type  2  (Rogers,  2014;  R.G.  Eggleston,  personal 
communication,  Nov  14,  2014).  Furthermore,  in  working  towards  a  Theory  of 
Consciousness  Awareness  the  unexpected  query  (UQ)  as  well  as  expected  query  (EQ) 
based  on  environmental  stimuli  will  need  to  be  research  based  on  environmental  stimuli 


(Rogers,  2014). 
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Conclusions 

In  this  chapter,  the  limitations,  ethieal  dimensions,  implieations,  and 
recommendations  of  the  study  were  presented.  The  limitations  of  the  study  included  the 
near-random  sample  and  quasi-experimental  quantitative  method.  Ethical  aspects 
included  the  confidentiality  of  the  partieipants,  and  meeting  the  requirements  stipulated  in 
the  Family  Edueational  Rights  and  Privacy  Act  (Family  Educational  Rights,  2010)  and 
HIPAA  (Creswell,  2009).  This  study  did  solve  the  problem. 

This  researeh,  due  to  the  seientifie  exigeney,  diseovered  the  preeise  and  predietive 
nexuses  for  current  theories  of  situation  awareness  and  concrete  quantifiable  eognitive 
and  perceptual  proeesses.  Previous  models  of  situation  awareness  were  still  conceptual 
models  that  provided  little  speeificity  with  regards  to  the  neurocognitive  processes  that 
are  neeessary  for  the  formation  and  maintenance  of  situation  awareness.  While  the 
TMSA  reeognizes  pereeption  as  a  eritical  first  step,  there  were  no  speeific  or  quantitative 
links  between  perceptual  abilities  and  situation  awareness;  nor  did  subsequent  work  prior 
to  this  dissertation  been  able  to  elarify  the  issue,  e.g.  the  effects  of  visual  ability  on  the 
level  of  situation  awareness.  Needed  theoretieal  advaneement  in  this  area  was  hampered 
by  a  lack  of  specific,  testable  predictions  regarding  plausible  eomponent  proeesses;  there 
was  little  theoretical  progress  due  to  this. 

The  three  researeh  questions  and  hypotheses  were  tested,  and  the  predictors 
assessed  were  found  to  be  statistically  significant.  As  a  result,  all  of  the  alternative 
hypotheses  that  indicated  signifieant  differences  were  aeeepted,  and  all  of  the  null 
hypotheses  were  rejeeted.  Visual  attentiveness,  visual  pereeptiveness,  and  visuospatial 
working  memory  were  and  are  predictors  of  SA.  To  encapsulate,  at  the  completion  of 
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this  field  experiment,  the  data  were  analyzed  and  the  tests  were  statistieally  signifieant  for 
the  three-predietor  visual  abilities  Vp,  Va,  and  Vswm  as  task  diffieulty  was  varied. 

While  filling  in  a  gap  in  the  TMS A  speeifieally  at  Level  1 ,  this  study  provides 
Theory  as  Law  to  the  existing  Theory  as  Narrative  TMSA.  This  dissertation  was 
sueeessful  in  filling  the  erueial  gap  in  the  TMSA  and  offers  an  Enhanced- TMSA  by 
providing  the  specifie  measurable  neuroeognitive  attributes  (Va,  Vp,  Vswm)  that  are  the 
visual  abilities  that  are  required  to  feed  Level  1  S A  -  pereeption,  allowing  SA  to  be 
predictable  (see  Figure  17). 


Figure  17.  Adding  Theory  as  Law  to  the  existing  Theory  as  Narrative  TMSA,  by 
successfully  filling  a  gap  in  the  TMSA  offering  an  Enhanced-TMSA. 
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In  conclusion,  this  dissertation  successfully  addressed  a  fundamental  problem  in 
existing  psyehologieal  theory,  expanded  the  understanding  of  situation  awareness, 
enhanced  the  Theoretical  Model  of  Situation  Awareness  (Figure  17),  and  provided 
practieal  reeommendations  to  enhance  safety  and  improve  human  performance.  This 
affects  society  worldwide.  This  study  resonated  scientific  inquiry  and  discovery.  While 
mueh  remains  to  be  addressed  in  this  area  by  future  work,  these  results  are  a  significant 
step  towards  a  complete  understanding  of  this  essential  aspeet  of  human  psychology. 
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Appendix  A;  List  of  Acronyms 
AFFSA  -  Air  Force  Flight  Standards  Agency 
AFIT  -  Air  Force  Institute  of  Technology 
AFLCMC  -  Air  Force  Life  Cycle  Management  Center 
AFRL  -  Air  Force  Research  Laboratory 
ANG  -  Air  National  Guard 
AS  -  Airlift  Squadron 
AW  -  Airlift  Wing 

BIFOV  -  Binocular  Instantaneous  Field  of  View 

CC  -  Commander 

CDEP  -  Cockpit  Design  Eye  Point 

CEIT  -  Controlled  Plight  Into  Terrain 

EASA  -  European  Aviation  Safety  Agency 

PEG  -  Electroencephalography 

ECG  -  Electrocardiogram 

EP  -  Exit  pupil 

ER  -  Eye  relief 

PAA  -  Pederal  Aviation  Administration 

PERPA  -  Pamily  Education  Rights  to  Privacy  Act 

POV  -  Pield  of  View 

PSP  -  Plight  Safety  Poundation 

HDD  -  Head  down  display 

HIPAA  -  Health  Insurance  Portability  and  Accountability  Act 
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HUD  -  Head  up  display 
HMD  -  Helmet  mounted  display 
HEMB  -  HUD  Eye  Motion  Box 
ID  -  Information  Display 

IV A+  -  Integrated  Visual  and  Audio  Performance  Test  Plus 
EOS  -  Eine-of-Sight 

NTSB  -  National  Transportation  Safety  Administration 

N-Back  Test  -  Number  Back  (Spatial  Working  Memory  Test  by  set  number  “N”) 
WE  -  Workload  (Cognitive  /  Mental) 

P  -  Pilot 

PEI  -  Primary  Plight  Information 

PRP  -  Psychological  Refractory  Period 

RT  -  Reaction  Time 

SA  -  Situation  Awareness 

SWM  -  Speed-working  memory 

USAP  -  United  States  Air  Porce 

VA  -  Visual  Attention 

Va  -  Visual  Attentiveness 

Vp  -  Visual  Perceptiveness 

Vswm  -  Visual  or  Visio-  Spatial  Working  Memory 


WM  -  Working  Memory 
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Appendix  B;  Instruments,  Procedures,  Selected  Data  /  Results,  and  Cert.  Letters 
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Histogram 


SA  Histogram 


Coefficient  Correiations^ 


Model 

Vswm 

Va 

Vp 

1  Correlations  Vswm 

1.000 

-.314 

-.503 

Va 

-.314 

1.000 

-.113 

Vp 

-.503 

-.113 

1.000 

Covariances  Vswm 

2.129E-5 

-3.834E-6 

-5.853E-6 

Va 

-3.834E-6 

6.983E-6 

-7.552E-7 

_ _ 

-5.853E-6 

-7.552E-7 

6.354E-6 

a.  Dependent  Variable:  SA 
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Collinearity  Diagnostics^ 


Model  Dimension 

Eigenvalue 

Condition  Index 

Variance  Proportions 

(Constant) 

Vp 

Va 

Vswm 

1  1 

3.910 

1.000 

.00 

.00 

.00 

.00 

2 

.048 

9.050 

.05 

.72 

.23 

.00 

3 

.029 

11.639 

.37 

.07 

.76 

.05 

4 

.014 

17.009 

.57 

.20 

.01 

.95 

a.  Dependent  Variable:  SA 


SA 

y  = -0.0095X  +  3.2674 

3.5  - -  =0.6655 


Vp,  r  =  0.82 


♦  SA 

- Linear  (SA) 


0.5  - 
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3.5 

3 

2.5 

2 

1.5 

1 

0.5 

0 


Improved  SA 


— I - 

20 


I 

40 


— I — 

60 


SA 


— I — 

80 


— I — 

100 


120 


V  =  -0.0054x  + 2.9286 
R*  =0.1616 

Va,  r=0.4 


♦  SA 


-  Linear  (SA) 


Va 


140 


160 


3.5 


SA 


V  =  -0.0144x  + 3.6016 
R^  =0.497 


3 

2.5 

2 

1.5 

1 

0.5 


Vspwm, 
r  =  0.705 


♦  SA 

—  Linear  (SA) 


0 


0 
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Vp 
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Va 


Vp 
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Va 


Descriptive  Statistics 


Mean 

Std.  Deviation 

N 

SA 

2.4238 

.31492 

12 

Vp 

88.4167 

26.92568 

12 

Va 

93.2500 

23.38269 

12 

Vswm 

81.6667 

15.39382 

12 
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Coefficient  Correiations^ 


Model 

Vswm 

Va 

Vp 

1  Correlations  Vswm 

1.000 

-.314 

-.503 

Va 

-.314 

1.000 

-.113 

Vp 

-.503 

-.113 

1.000 

Covariances  Vswm 

2.129E-5 

-3.834E-6 

-5.853E-6 

Va 

-3.834E-6 

6.983E-6 

-7.552E-7 

Vp 

-5.853E-6 

-7.552E-7 

6.354E-6 

a.  Dependent  Variable:  SA 


Coiiinearity  Diagnostics^ 


Model 

Dime 

nsion 

Eigenvalue 

Conditio 

n  Index 

Variance  Proportions 

(Consta 

nt) 

Vp 

Va 

Vswm 

1 

1 

3.910 

1.000 

.00 

.00 

.00 

.00 

2 

.048 

9.050 

.05 

.72 

.23 

.00 

3 

.029 

11.639 

.37 

.07 

.76 

.05 

4 

.014 

17.009 

.57 

.20 

.01 

.95 

a.  Dependent  Variable:  SA 
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Statistics 


Vp 

Va 

Vswm 

SA 

N  Valid 

12 

12 

12 

12 

Missing 

0 

0 

0 

0 

Mean 

88.4167 

93.2500 

81.6667 

2.4238 

Std.  Error  of  Mean 

7.77278 

6.75000 

4.44381 

.09091 

Median 

98.0000 

101.0000 

88.5000 

2.4616 

Mode 

98.00 

45.00® 

92.00 

2.08® 

Std.  Deviation 

26.92568 

23.38269 

15.39382 

.31492 

Variance 

724.992 

546.750 

236.970 

.099 

Skewness 

-1.324 

-.972 

-1.126 

-.029 

Std.  Error  of  Skewness 

.637 

.637 

.637 

.637 

Kurtosis 

1.199 

.709 

.251 

-.153 

Std.  Error  of  Kurtosis 

1.232 

1.232 

1.232 

1.232 

Range 

91.00 

83.00 

48.00 

1.13 

Minimum 

28.00 

45.00 

50.00 

1.88 

Maximum 

119.00 

128.00 

98.00 

3.00 

Sum 

1061.00 

1119.00 

980.00 

29.09 

Percentiles  10 

34.0000 

48.3000 

52.1000 

1.9375 

20 

62.4000 

69.8000 

66.0000 

2.0833 

25 

74.5000 

81.2500 

72.5000 

2.1250 

30 

81.0000 

87.1000 

73.8000 

2.2333 

40 

95.6000 

95.2000 

83.0000 

2.3923 

50 

98.0000 

101.0000 

88.5000 

2.4616 

60 

100.4000 

104.4000 

90.8000 

2.5000 

70 

103.1000 

106.1000 

92.0000 

2.5843 

75 

103.7500 

106.7500 

92.0000 

2.6679 

80 

107.6000 

107.8000 

93.2000 

2.7057 

90 

117.2000 

122.3000 

97.1000 

2.9143 

a.  Multiple  modes  exist.  The  smallest  value  is  shown 


Frequency 


Percent 


Valid  Percent 


Cumulative 

Percent 


1 

8.3 

8.3 

8.3 

1 

8.3 

8.3 

16.7 

1 

8.3 

8.3 

25.0 

1 

8.3 

8.3 

33.3 

1 

8.3 

8.3 

41.7 

2 

16.7 

16.7 

58.3 

1 

8.3 

8.3 

66.7 

1 

8.3 

8.3 

75.0 

1 

8.3 

8.3 

83.3 

1 

8.3 

8.3 

91.7 

1 

8.3 

8.3 

100.0 

12 

100.0 

100.0 

Freauenc 


Percent 

Valid  Percent 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

8.3 

100.0 

100.0 

Cumulative 

Percent 


8.3 

16.7 

25.0 


33.3 


41.7 


50.0 


58.3 

66.7 


75.0 

83.3 

91.7 

100.0 


Frequency 


Percent 


Valid  Percent 


Cumulative 

Percent 
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Attachment  G:  Certification  of  Vitaport  Interference  Testing 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  AERONAUTtCAl  SYSTEMS  CENTER  (APMC) 
WRiGHT-PATTERSON  AIR  FORCE  BASE.  OHIO 


2  1  JUL199T 

MEMORANDUM  FOR:  AiyCHFP 

ATTN:  MR.  GLENN  WILSON 


FROM:  ASCTENAE 


SUBJECT:  Air  Force  Instruction  (AFI)  11-206  Electromagnetic  Interference  Certification 
of  the  Vitaport  2  Physiological  Recorder 


1.  The  Vitaport  2  Physiological  Recorder  has  been  tested  for  emissions  to  the  limits  of 
MIL-STD-461D  and  in  accordance  with  MIL-STD-462D  test  methods. 

Equipment  certification  per  AFI  11-206  is  as  follows: 

The  Vitaport  2  Physiological  Recorder  has  failed  to  meet  required  limits  for  radiated 
emissions.  The  Vitaport  2  Physiological  Recorder  shall  not  be  used  during  takeoff  and 
landing  thelow  10,000  ft. I.  or  whenever  directed  by  a  crew  member.  This  equipment  may 
be  used  at  other  times  (above  10, (XX)  ft.),  provided  the  pilot  and  crew  are  aware  that  the 
equipment  is  being  operated.  If  interference  from  the  Vitaport  2  Physiological  Recorder  is 
suspected,  the  crew  may  prohibit  its  operation. 

2.  The  Vitaport  2  Physiological  Recorder  can  be  modified  to  be  used  during  takeoff  and 
landing  (below  lO.OCX)  ft.)  by  installing  a  double  wrapped  Steward  P/N  28A2024-0A0 
Ferrite  located  7  cm  from  the  unit  on  the  interconnecting  cable.  If  the  modification  is 
incoiporated  the  equipment  may  be  used  during  all  phases  of  flight  with  the  same 
restriction  that  the  pilot  and  crew  are  aware  that  the  equipment  is  being  operated.  If 
interference  from  the  Vitaport  2  Physiological  Recorder  is  suspected,  the  crew  may 
prohibit  its  operation. 

3.  This  certification  docs  not  consider  the  effects  that  other  electronic  equipment  may 
have  on  the  Vitaport  2  Physiological  Recorder.  Radiated  and  conducted  emission  testing 
is  performed  on  your  equipment  solely  to  provide  reasonable  assurance  that  the  equipment 
will  not  cause  electromagnetic  interference  problems  with  existing  aircraft  equipment.  No 
testing  was  performed  to  asse.ss  whether  the  Vitaport  2  Physiological  Recorder  may  be 
adversely  affected  by  existing  aircraft  equipment.  Testing  of  your  equipment  was  limited 
to  emissions.  Since  we  did  not  evaluate  your  equipment  for  susceptibility,  the  potential 
for  degradation  of  your  equipment  due  to  operation  of  other  aircraft  systems  is  unknown. 
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Intentional  Blank 
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Attachment:  Bedford  Workload  Scale 

Workload  Description 


Was  workload 
satisfactory 
witiiout  reduction  in 
spare  (workload)  capacity? 


Was  workload  toteable 
for  the  task? 


NO 


a 

YES 

Was  it  possible  to 
complete  the  task? 


Pilot  Decisions 


“Rating” 


Workload  insignificant 

1 

Workload  low 

2 

Enough  spare  capacity  for  all 
desirable  additional  tasks 

3 

Task  abandoned.  Pilot  unable  to 
apply  sufficient  effort 


Insufficient  spare  capacity  for  easy 
att^ion  to  additional  tasks 

4 

Reduced  spare  capacity.  Additional 
tasks  cannot  be  given  the  desired 
amouth  of  attention 

5 

Little  spare  capacity:  level  of  effort 
allows  little  attention  to  additional 
tasks 

6 

Very  little  spare  capacity,  but 
maintenance  of  effort  in  the  primary 
tasks  not  in  question 

7 

Very  high  workload  with  almost  no 
spare  capacity.  Difficulty  in 
maintaming  level  of  effcal 

8 

Extremely  high  workload.  No  spare 
capacity.  Serious  doubts  as  to  ability 
to  maintain  level  of  effort 

9 

10 
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Attachment :  China  Lake  Situation  Awareness  Scale 

Instructions:  circle  the  number  that  best  matches  your  situational  awareness 


Simation  Awareness 
Scale  Value 

Content 

Veiy  Good  - 1 

•  Full  knowledge  of  A/C  energy  state  /  tactical  enviiomnent  mission 

•  Full  ability  to  anticipate  /  acconmiodate  trends 

Good  -  2 

•  Full  knowledge  of  A/C  energy  state  tactical  em'iromnent  /  mission 

•  Partial  ability  to  anticipate  /  acconmiodate  trends 

Adequate  -  3 

•  Full  knowledge  of  A  C  energy  state  tactical  enviiomnent  mission 

•  Satiuated  ability  to  anticipate  /  acconmiodate  trends 

•  Some  shedding  of  minor  tasks 

Poor  -  4 

•  Fail'  knowledge  of  A/C  energy  state  '  tactical  em  iroimient  /  mission 

•  Satiuated  ability'  to  anticipate  /  acconmiodate  trends 

•  Shedding  of  all  minor  tasks  as  well  as  many  not  essential  to  flight  safety  / 
mission  effectiveness 

Veiy  Poor  -  5 

•  Minimal  knowledge  of  A/C  energy'  state  /  tactical  em  u  oimient  mission 

•  Ox  ersatiuated  ability  to  anticipate  /  acconmiodate  trends 

•  Shedding  of  all  tasks  not  absolutely  essential  to  fliglit  safety  /  mission 
effectiveness 

Note;  A/C  -  aiicraft 

229 


Attachment :  I VA+ information 


IVA+Plus  (formerly  known  as  the  IVA)  is  a  unique 
combined  auditory  and  visual  continuous  performance 
test  of  attentional  functioning,  originally  developed  as 
a  diagnostic  aid. 

The  IVA+Plus  main  test  task,  which  lasts 
approximately  thirteen  minutes,  presents  500  trials  of 
"l"s  and  "2"s  in  a  pseudo-random  pattern  requiring 
the  shifting  of  sets  between  the  visual  and  auditory 
modalities.  The  subject  is  required  to  click  the  mouse 
only  when  he  sees  or  hears  a  "1"  and  to  inhibit 
clicking  when  he  sees  or  hears  a  "2."  During  some 
segments  of  the  IVA+Plus  test,  the  "l"s  are  more 
common  than  the  "2"s,  creating  a  response  set  which 
"pulls"  for  errors  of  commission,  or  impulsivity.  During  alternate  segments  of  the  IVA+Plus 
test,  the  "l"s  occur  rarely;  this  invites  more  errors  of  omission,  or  inattention,  since  the 
subject  must  remain  vigilant  while  he  waits  for  a  "1"  to  occur. 

IVA+Plus'  normative  group  (N  =  1700)  is  divided  by  gender,  and  grouped  by  age  as 
follows:  6,  7,  8,  9,  10,  11,  12,  13,  14,  15,  16,  17-18,  19-21,  22-24,  25-29,  30-34,  35- 
39,  40-44,  45-54,  55-65,  66-96.  The  database  was  primarily  collected  in  Richmond, 

Texas,  Michigan,  California  and  Florida.  All  individuals  were  excluded  who  were  in  therapy, 
had  a  history  of  LD,  hyperactivity  or  attention  problems,  who  were  on  any  type 
of  medication  (otherthan  birth  control  unless  >55  years  of  age),  who  had  a  history  of 
neurological  problems  (dementia,  stroke  orTBI)  and  those  who  could  not  validly  complete 
the  test.  A  relatively  equal  number  of  males  and  females  were  included  in  each  age/sex 
group  and  an  effort  was  made  to  have  about  30  males  and  30  females  in  each  age 
group  though  this  was  not  always  possible.  At  about  30  in  each  group,  the  standard 
error  ranged  between  3-4  points  on  a  standard  scale  for  all  the  IVA+Plus  scales.  Many 
different  ethnic  groups  were  included  in  the  normative  sample,  but  this  data  has  not  been 
broken  down.  The  normative  data  is  available  in  the  IVA+Plus  program  sub-  directory 
and  can  be  used  in  most  cases  to  manually  calculate  the  standard  scores, 
except  when  the  standard  deviation  is  small  and  the  percent  raw  score  reported  in  the 
reports  has  been  rounded  before  being  displayed  (mainly  an  issue  with  young  adults  who 
make  few  errors.) 

Scoring  of  the  test  includes  reaction  time  and  accuracy  measures,  compounded  into 
scores  for  each  of  the  test  segments. 
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BRUCE  E.  KLINE,  PSY.D.  AND  ASSOCIATES 

CLINICAL  PSYCHOLOGISTS 


529  E.  Stroop  Rd 
Kettering,  OH  45429 


Telephone 
(937)  294-6004 


June  7,  201 1 

Dr.  James  C.  Christensen,  Research  Psychologist 

Applied  Cognitive  Neuroscience  Branch 

71 1  HPW/RHCP 

Air  Force  Research  Laboratories 

Building  33  WPAFB,  OH  45433 

RL:  Steven  D.  Harbour  (Ph  D,  Psychology  Candidate) 

Use  of  the  Integrated  Visual  and  Auditory  performance  Plus  {rVA+)  Test 
Letter  of  Recommendation 

Dear  Dr.  Christensen; 

Mr.  Steve  Harbour  has  been  performing  his  pre-doctoral  internship  program,  while  studying  at 
Northcentral  University,  in  my  clinic  for  the  past  2  years.  He  has  logged  more  than  1000  clinical 
hours  under  my  supervision.  After  completing  PSY7105  Psychological  Test  and  Measures  with 
the  grade  of  ‘A’,  Steve  has  scored  and  interpreted  multiple  rVA+  tests  that  he  has  given  to  our 
patients  and  clients.  Mr.  Harbour  is  highly  capable,  competent,  and  skilled  at  running  the  IVA. 

It  is  with  great  pleasure  that  I  highly  recommend  Mr.  Harbour  as  being  fully  qualified  to 
administer,  score,  and  interpret  the  results  of  the  Integrated  Visual  and  Auditory  performance 
Plus  Test.  Steve  has  my  strongest  recommendations-  as  he  will  perform  at  the  hipest  level.  He 
came  to  me  as  a  USAF  Test  Pilot,  Research  Human  Factors  Engineer,  and  a  Ph  D  in  Psychology 
student.  Steve  has  been  performing  extensive  research  in  the  area  of  Cognitive  Neuroscience  and 
human  systems  integration  (workload  and  vigilance).  His  research  focuses  on  Post  Traumatic 
Stress  Disorder  and  Traumatic  Brain  Injuries  and  their  effects  on  cognition  and  memory  while 
investigating  and  testing  new  and  innovated  ways  of  treatment. 

Whether,  it  is  behind  the  controls  of  an  aircraft  as  a  USAF'  Test  Pilot,  or  a  Human  Factors 
Engineer  developing  new  cockpits  for  both  piloted  and  unpiloted  vehicles,  Steve  is  highly 
experience  performing  Human  Factors  and  Psychological  research. 
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Scientists  used  a  modified  version  of  the  n- 
back  test,  which  is  schematized  above,  to 
achieve  gains  in  fluid  intelligence  previously 
thought  impossible.  The  image  shows  how 
users  were  forced  to  remember  both  visual 
and  auditory  information  streams. 

Courtesy  Martin  Buschkuehl 


Visuospatial  Working  Memory  (IV).  This  is  working  memory  that  contains 
visual  and  spatial  information  that  is  stored  in  the  visuospatial  sketchpad  in  the  mind 
(Baddeley,  2006),  this  immediate  memory  can  be  thought  of  as  a  workbench  where 
material  is  continuously  being  combined  and  transformed.  Visuospatial  working 
memory  will  be  assessed  via  the  spatial  n-  back  task  (Gevins  &  Cutillo,  1993), 
explained  in  detail  in  subsequent  section  along  with  validity.  This  variable’s  level  of 
measurement  is  ratio  and  is  a  percent  of  selections  that  are  correct  out  of  the  total 
number;  an  example  could  be  85%.  Scores  can  range  from  0%  correct  to  100% 
correct. 

The  Visuospatial  Working  Memory  test  is  a  two-back  forced  choice 
computerized  test  paradigm  in  which  the  participant  is  presented  with  a  black  X,  which 
can  appear  at  any  of  five  different  locations  on  the  screen  (AFRL,  2009;  Gevins  & 
Cutillo,  1993;  Sohn,  &  Doane,  1997,  2000,  2003).  Participants  observe  a  sequence  of 
presentations  of  the  X  at  the  different  locations;  their  task  is  to  compare  the  current 
location  of  the  X  to  where  the  X  appeared  two  presentations  prior.  This  is  done  on  a 
continuous  basis,  requiring  the  participants  to  hold  the  two  previous  presentation 
locations  in  spatial  working  memory.  This  test  is  repeated  multiple  times,  over  five 
minutes.  The  spatial  n-back  task  is  a  test  of  the  participant’s  ability  to  retain  spatial 
information  and  to  manipulate  remembered  items  in  working  memory  (AFRL,  2009; 
Sohn,  &  Doane,  1997,  2000,  2003).  It  is  reliable  across  people  and  studies  (Owen, 
McMillan,  Laird,  &  Bullmore,  2005). 


OHIO  AIR  NATIONAL  GUARD 

HEADQUARTERS  179TH  AIRLIFT  WING  (AMC) 

MANSFIELD.  OHIO 


MEMORANDUM  FOR  179AW  PERSONNEL 
FROM;  179AW/CC 

SUBJECT;  A  letter  of  support  for  Research  by  the  Principle  Investigators  James  C.  Christensen 
(AFRL)  and  Steven  D.  Harbour  (ASC) 


Mansfield  Air  National  Guard  Base,  179th  Airlift  Wing  will  be  available  for  recruitment  of 
participants  for  the  research  Titled:  C-27J  Cockpit  and  HUD  Work  Load  (WL)  and  Situation 
Awareness  (SA)  Study.  The  purpose  is  HUD  usability  testing  and  global  cockpit  workload.  The 
persons  conducting  the  study  are;  James  C.  Christensen,  Steven  D.  Harbour,  Justin  Estepp,  ILt 
Tyron  Gray,  Glenn  Wilson,  Iris  Davis,  and  Margaret  Funke.  The  proposed  study  may  provide 
valuable  information  about  C-27J  Cockpit  and  HUD  Work  Load  (WL)  and  Situation  Awareness 
(SA),  HUD  usability,  global  workload,  viability  of  objective  and  subjective  measures,  and 
correlations  and  causations. 

We  will  allow  for  recruitment  of  crewmembers  for  the  research  at  our  unit  following  IRB 
approval.  The  project  will  only  be  scheduled  for  missions  that  include  1  to  2  participants  that 
have  consented  to  participate  in  the  research  per  sortie.  Pilots  will  also  be  asked  to  perform  the 
IVA  test  (a  simple  13  to  20  minute  computer  based  test)  at  their  convenience  with  study 
personnel.  Objective  measures  will  occur  during  specific  points  in  the  mission:  Low  Level, 
Airdrop,  Tactical  landings  and  takeoffs  (assaults).  As  well  as  Approaches  (prec/non-prec)  and 
Touch  -  n  -  go’s,  and  take  offs  and  landings.  The  missions  will  remain  unchanged.  After 
landing  and  during  post  flight  the  crewmember  will  also  be  asked  to  fill-out  some  surveys. 

These  objective  SA  and  WL  measures  could  provide  more  efficient  and  effective  results  to  the 
senior  leader  decision  makers  while  making  the  C-27J  a  more  effective  and  safe  platform. 

Sincerely, 


MCCUE,  Colonel,  Ohio  ANG 
179'’’  Airlift  Wing  Commander 
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Appendix  C:  NCU  IRB  Approval 


Student’s  name:  Steven  Harbour 
School  of  Psychology 
Dear  Steven, 

Thank  you  for  your  submission  of  your  IRB  application  and  supporting  documents  to  IRB.  Please 
review  the  feedback  provided  to  you  regarding  your  responses  to  the  IRB  application  and  other 
supporting  documents. 

This  is  an  expedited  IRB  review. 

Purpose  and  Significance  section 

No  comments 

Participation  Population  and  Recruitment  section 

No  comments 

Research  Procedure  section 

No  comments 

Risks  and  Benefits  section 

No  comments 

Informed  Consent  land  Assent)  section 

No  comments 

Anonymity  or  Confidentiality  section 

No  comments 

Audio/Video  Taping  section 

No  comments 
Compensation  section 

No  comments 
Deception  section 

No  comments 
Debriefing  section 

No  comments 
Additional  Comments 

No  comments 
Supporting  Documents 

IRB  obtained  from  USAF 
Decision  Status:  Approve 

Good  luck  with  data  collection.  Be  sure  to  keep  in  close  communication  with  your  mentor  and 
dissertation  committee.  Keep  in  mind  that  if  there  are  any  changes  to  the  research  procedures,  you 
must  notify  the  IRB. 

Sincerely, 


Alice  Yick,  Ph.D. 

NCU,  Associate  Director  of  IRB  and  IRB  Reviewer 

Reference:  Steven  Harbour 
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IRB:  2014-08-19-263 

Dear  Dr.  James  C  Christensen,  Dissertation  Chair: 

Northcentral  University  approved  Steven's  research  project  entitled,  A 
Neuroergonomic  Quasi-Experiment:  Predictors  of  Situation  Awareness  and  Display 
Usability  with  USAF  Pilots  while  Performing  Complex  Tasks. 

As  an  investigator  of  human  subjects,  the  student  researcher's  responsibilities 
include  the  following: 

1.  Report  promptly  proposed  changes  in  previously  approved  IRB  to  your 
study  such  as  changes  to  the  sampling  design,  research  procedures, 
consent/assent  forms  and  any  other  study  documents,  regardless  of 
how  minor  the  proposed  changes  might  be.  (Review  the  modifications 
request  procedures  in  the  Dissertation  Center,  under  the  IRB  thread). 

2.  Report  promptly  to  the  IRB  any  injuries  or  other  unanticipated  or 
adverse  events  involving  risks  or  harms  to  human  research  subjects  or 
others. 

3.  Report  to  the  IRB  the  study's  closing  (i.e.,  completion  of  data  collection 
and  data  analysis).  Note  the  above  expiration  date  of  the  IRB 
approvai.  It  is  the  researcher's  responsibility  to  report  the  closing  of 
the  study  to  the  IRB  before  the  study's  expiration  date.  (Form  is  in  the 
Dissertation  Center,  under  the  IRB  thread). 

4.  If  the  study  is  to  continue  past  the  expiration  date,  student  researcher 
must  submit  a  request  for  continuing  review  prior.  Note  the  above 
continuing  review  due  date.  It  is  the  researcher's  responsibility  to 
obtain  re-approval  from  the  IRB  before  the  study's  expiration  date. 

(Form  is  in  the  Dissertation  Center,  under  the  IRB  thread). 

5.  If  re-approval  for  continuing  review  is  not  obtained  (unless  the  study 
has  been  reported  to  the  IRB  as  closed)  prior  to  the  expiration  date,  all 
activities  involving  human  subjects  and  data  analysis  must  cease 
immediately 

Sincerely, 

Dr.  Alice  Yick 

NCU,  Associate  Director  of  IRB  and  IRB  Reviewer 
Northcentral  University 
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Appendix  D;  USAF  IRB  Approved 

A  Neuroergonomic  Quasi-Experiment:  Predictors  of  Situation  Awareness  and  Display 
Usability  with  USAF  Pilots  while  Performing  Complex  Tasks 
USAF  IRB  Approved 
FWRH 


1,  Principal  Investigator 

Steven  Flarbour,  Fluman  Factors  Engineer,  NCU  Doctoral 
Candidate  ASC/WLNJ,  937-255-8468 

Steven.harbour@wpafb.af.mil 

2,  Associate  Investigator 

a.  Dr.  James  C.  Christensen,  Ph.D.,  DR-II,  Research  Psychologist,  NCU 
Dissertation  Chair  711  HPW/RHCP,  937-938-3603 
james.christensen@wpafb.af.mil 

3,  Medical  Consultant  or  Monitor 

William  P.  Butler,  Col,  USAF,  MC,  CFS;  711  HPW/IR;  937-656-5437; 

william.butler3@wpafb.af.mil 

4,  Facility/Contractor 

Data  collection  will  take  place  with  the  179‘^  AW,  Mansfield,  OFI.  Analysis  and  storage 
will  take  place  in  B840,  WPAFB,  OFI.  Contracting  support  will  be  provided  by  Ball 
Aerospace  under  contract  number  FA8650-08-D-6801 .  Under  the  Ball  Aerospace  DoD 
Addendum  F50343. 

5,  Objective 

Despite  the  scientific  exigency  there  are  no  precise  and  predictive  nexuses  for  current 
theories  of  situation  awareness  and  specific  quantifiable  cognitive  and  perceptual 
processes.  Current  models  of  situation  awareness  are  still  conceptual  models  that 
provide  little  specificity  with  regards  to  the  neurocognitive  processes  that  are  necessary 
for  the  formation  and  maintenance  of  situation  awareness.  While  the  Theoretical  Model 
of  Situation  Awareness  (TMSA)  recognizes  perception  as  a  critical  first  step,  there  are  no 
specific  or  quantitative  links  between  perceptual  abilities  and  situation  awareness;  nor 
has  subsequent  work  been  able  to  clarify  the  issue,  e.g.  the  effects  of  visual  ability  on 
the  level  of  situation  awareness.  Needed  theoretical  advancement  in  this  area  continues 
to  be  hampered  by  a  lack  of  specific,  testable  predictions  regarding  plausible  component 
processes;  there  has  been  little  theoretical  progress  due  to  this.  Seminal  research  found 
that  visual  processing  was  involved  in  situation  awareness.  However,  the  specific 
component  processes  have  not  been  explored  in  detail.  Consequently,  the  primary 
purpose  of  this  quasi-experimental  quantitative  study  is  to  test  the  predictive  value  of 
these  specific  candidate  visuo-cognitive  abilities  Visual  Attentiveness,  Visual 
Perceptiveness,  and  Visuospatial  Working  Memory  as  predictors  of  situation  awareness. 
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e.g.  the  effects  of  visual  ability  on  the  level  of  situation  awareness.  Therefore,  this 
project  will  test  the  predictive  value  of  these  variables  as  factors  between  a  particular 
task  to  be  performed  and  the  eventual  outcome  of  situation  awareness.  As  Pavlov, 
Watson,  and  Skinner  performed  field  experiments  in  order  to  contribute  to  psychological 
theory,  a  field  experiment  in  the  paradigm  of  neuroergonomics  will  be  used  for  this 
study.  The  results  of  this  study  will  fill  in  a  key  gap  in  the  TMSA,  and  in  so  doing  enable 
both  theoretical  refinement  and  practical  applications  such  as  improved  procedures, 
training  for  pilots,  and  display  design  that  improve  flight  safety.  Additionally,  this 
research  is  to  evaluate  the  impact  of  the  C-27J  HUD  configuration  on  workload  and 
situation  awareness  (SA),  and  to  discover  human  antecedents  of  SA,  while  testing  the 
usability  of  three  alternative  heads-up  displays  (HUDs)  in  the  Alenia  C-27J.  Publicly 
released  information  states  that  the  C-27J  is  still  undergoing  testing  with  the  first  four 
aircraft  having  been  delivered  to  the  179**^  Airlift  Wing  (Ohio  ANG),  Mansfield,  Ohio; 
however,  at  least  some  of  those  aircraft  are  expected  to  be  deployed  yet  this  summer. 

6,  Background 

Based  on  testing  conducted  by  179**^  AW  personnel  and  the  C-27J  Systems  Program 
Office  (SPO),  the  HUD  as  currently  configured  in  the  aircraft  is  seriously  misaligned  with 
the  cockpit  design  eye  point/eye  movement  box  (DEP/EMB).  If  a  pilot  correctly  positions 
themselves  in  the  DEP  using  the  design  eye  spheres  on  the  glare  shield,  they  are 
approximately  2  inches  below  the  optimal  position  to  view  HUD  symbology.  As  a  result, 
the  lower  portion  of  the  HUD  is  not  visible,  with  the  missing  portion  ranging  from  a 
minimum  of  25%  of  the  screen  to  a  maximum  of  100%,  depending  on  pilot  height  and 
exact  position.  Raising  the  seat  above  the  DEP  corrects  HUD  visibility;  however  when 
raised  sufficiently  to  view  the  HUD,  the  pilot’s  body  obstructs  control  yoke  travel.  A 
significant  majority  of  the  available  C-27J  pilots  report  at  least  30%  of  aft  control 
authority  lost,  and  at  least  30%  of  bank  authority  with  aft  deflection.  This  loss  of  yoke 
travel  is  a  critical  issue  for  tactical  airlift  missions.  Likewise,  the  loss  of  some  or  all  HUD 
symbology  is  a  critical  safety  issue,  particularly  in  night  operations.  ASC  safety  review 
has  consequently  designated  the  HUD/yoke  travel  hazard  category  1 A  - 
catastrophic/frequent/high. 

Two  proposed  solutions  are  being  prepared  by  the  Aeronautical  Systems  Center;  the  first 
is  to  simply  disable  the  HUD,  while  the  second  is  to  use  spacers  to  lower  the  HUD 
approximately  2  inches.  The  following  warning  is  currently  provided  to  pilots:  “If  the  pilot 
is  unable  to  view  all  of  the  HUD  symbology  without  obstructing  the  ability  to  aviate  the 
aircraft,  then  the  pilot  shall  not  use  the  HUD  for  flight.”  Lowering  the  HUD  does  reduce 
forward  visibility  out  of  the  windscreen.  In  order  to  make  a  fully  informed  decision 
regarding  this  issue,  ASC  has  requested  that  AFRL  assist  with  usability  testing  on  both 
of  these  options,  as  well  as  the  baseline/original  HUD.  With  deployment  anticipated  this 
summer,  ASC  is  requesting  rapid  response  from  AFRL.  In  collaboration  with  the  SPO, 
AFRL  is  proposing  to  conduct  usability  testing  in  association  with  training  flights 
conducted  by  179  AW  pilots. 


The  testing  will  consist  of  standard  subjective  inventories  (Bedford  Workload  Scale, 
Roscoe  and  Ellis,  1990;  China  Lake  Situation  Awareness  Scale,  Adams,  Kane,  and 
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Bates,  1998;  Attachments  b  and  c),  standard  cognitive  testing  (integrated  visual/auditory 
continuous  performance  test,  IVA+  and  N-back  visio-working  memory  test;  Attachment 
d),  and  electrocardiographic  (EKG)  analysis  of  heart  rate  and  heart  rate  variability,  and 
electroencephalogram  (EEG)  during  flight.  Subjective  scales  have  been  widely  used  to 
assess  workload;  however,  the  SPO  is  concerned  that  limiting  data  collection  to 
subjective  measures  may  result  in  skewed  or  uninformative  results  due  to  deployment 
pressure.  As  a  result,  the  subjective  data  will  be  complemented  with  objective 
physiological  data  that  is  sensitive  to  workload  differences,  specifically  heart  rate  and 
heart  rate  variability  (Hankins  and  Wilson,  1998;  Nickel  and  Nachreiner,  2003).  The 
purpose  of  the  IVA+  (Turner  and  Sandford,  1995)  is  to  also  assess  attention  switching 
and  cognitive  performance;  one  of  the  primary  consequences  of  disabling  or  using  the 
original  HUD  will  be  increased  workload  associated  with  rapid  attention  switching 
between  the  primary  flight  display  (heads-down  display,  HDD)  and  the  view  outside  the 
cockpit.  By  including  one  quick  test  of  basic  cognitive  ability,  we  will  have  a  covariate 
that  should  aid  in  interpreting  the  results,  e.g.  by  enabling  the  binning  of  data  based  on 
attention  switching  performance. 

7,  Impact 

As  a  category  1 A  safety  hazard,  the  C-27Js  will  not  deploy  until  adequate  resolution  has 
been  reached.  This  research  will  quantify  the  impact  of  disabling  or  lowering  the  HUD,  in 
terms  of  subjective  workload  and  situation  awareness,  and  objective  workload.  The 
results  of  this  research  will  enable  a  go/no  go  decision. 

8,  Experimental  Plan 

a.  Equipment: 

The  Bedford  Workload  and  China  Lake  SA  scales  will  be  delivered  via  paper 
questionnaires.  The  IVA  continuous  performance  test  will  be  delivered  via  laptop 
computer  provided  by  AFRL  personnel.  The  electrocardiographic  data  will  be  collected 
using  a  Vitaport  system  (Temec  Instruments  B.V.,  Kerkrade,  Netherlands),  which  is  a 
small,  portable  pilot-worn  physiological  data  collection  system  with  onboard  digital  data 
storage  (e.g.  approved  protocol  FWR20100077H).  This  device  has  been  certified  for  use 
in  all  flight  phases  subject  to  aircrew  consent  (Attachment  G,  Section  2;  the  ferrite  core 
wrapper  will  be  used  at  all  times).  Sterile,  single-use  ECG  leads  will  be  placed  on  the 
sternum  and  clavicle  and  EEG  leads  will  be  placed  on  the  scalp  in  accordance  with 
standard  test  procedures. 

b.  Subjects: 

a.  Source:  All  participants  will  be  rated  C-27J  ANG  pilots  on  active  flight 
status  at  Mansfield,  OH.  Approximately  25  assigned  pilots  currently 
meet  those  criteria. 

b.  Number  required:  Based  on  the  AFRL  team’s  previous  experience  conducting 
workload  studies,  approximately  12  individuals  will  be  required  to  achieve  adequate 

statistical  power. 

c.  Inclusion/exclusion/age  range:  To  participate,  pilots  must  be  rated  C-27J  pilots,  on 
active  flight  status,  and  currently  flying  training  missions.  There  are  no  additional 
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requirements  or  limitations  on  participation. 
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c.  Duration: 

This  testing  will  take  no  more  than  approximately  365  days  from  approval 
(planned  to  be  completed  in  15  to  30  calendar  days  from  start),  and  is  driven  by 
ASC  requirements  (after  NCU  IRB  approval).  Some  additional  follow  up 
analyses  may  be  conducted. 

d.  Description  of  experiment,  data  collection,  and  analysis: 

This  project  will  piggyback  on  normal  training  sorties  conducted  by  the  179  AW; 
research  personnel  will  not  direct,  schedule,  or  otherwise  interfere  in  training 
missions.  While  a  variety  of  sorties  are  being  flown,  the  sorties  determined  to 
be  of  greatest  interest  based  on  179  AW  input  are  the  following: 

1.  Short-field  take-off  and  landing,  day 

2.  Short-field  take-off  and  landing,  night 

3.  Low  level  air  drop,  day 

4.  Low  level  air  drop,  night 

We  will  endeavor  to  obtain  one  sortie  of  each  type  from  12  pilots  total.  Ideally, 
this  will  be  crossed  with  HUD  types:  no  HUD  (HDD),  and  the  moved/new  HUD, 
resulting  in  a  total  of  approximately  8  sorties  equivalent  per  pilot  (2  or  more 
pilots  per  sortie).  We  understand  and  expect  that  this  ideal  will  not  be 
guaranteed  achievable  with  all  pilots  and  mission  constraints.  Order  and  time 
between  sorties  will  not  be  controlled,  to  avoid  any  interference  with  training 
activities. 

Pilots  will  be  briefed  as  to  the  purpose  of  the  study,  and  then  given  an  invitation 
memo  (Attachment  f).  Pilots  who  wish  to  participate  will  complete  comprehensive 
written  informed  consent. 

Data  collection  will  commence  with  preflight  delivery  of  the  IVA+  (once  per 
pilot),  the  2-Back  spatial  working  memory  test,  and  placement  of  the  ECG 
leads  on  sternum  and  clavicles  and  EEG  leads  on  scalp  (once  per  day/sortie). 
Female  pilots  will  be  given  the  opportunity  to  place  their  own  leads  in  private 
following  brief  researcher  instruction. 

The  Vitaport  data  acquisition  box  will  be  activated  after  placing  the  leads,  and 
left  on  for  the  duration  of  the  sortie.  Preflight  data  will  be  used  to  establish 
individual  baselines  for  each  flight.  Timestamps  will  be  taken  during  each  phase 
of  the  sortie  to  enable  analyses;  the  Vitaport  will  be  deactivated  during  removal 
of  the  leads  postflight. 

During  sortie  debriefing,  each  pilot  will  be  asked  to  complete  one  Bedford  and 
one  China  Lake  scale,  with  instructions  to  consider  the  HUD  or  lack  thereof  or 
the  HDD. 
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Analysis  of  subjective,  IVA+  performance,  N-back,  and  ECG  /  EEG  data  will 
include  within  subjects  repeated  measures  analyses  of  variance  (ANOVA), 
followed  by  analyses  of  simple  main  effects,  interactions  and  multiple 
comparisons.  We  are  especially  concerned  with  testing  for  significant 
differences  among  the  HUD  and  HDD  conditions. 

Safety  monitoring: 

All  flight  safety  issues  are  the  responsibility  of  the  appropriate  Safety  Review  Board  and 
squadron  SOF.  Researchers  will  be  responsible  for  monitoring  participants  for  any 
adverse  reactions  possibly  associated  with  electrode  placement,  such  as  urticarial  (hives) 
or  contact  dermatitis.  In  such  an  event,  the  medical  monitor  would  be  immediately 
notified. 

Confidentiality  protection: 

In  order  to  avoid  the  collection  of  direct  or  indirect  personally  identifiable  information,  there 
will  be  no  demographic  data  collected.  Each  pilot  participant  will  receive  a  randomly- 
generated,  non-sequential  numeric  identifier  that  will  be  used  to  identify  their  data. 
Individual  data  will  include  subjective  responses  to  survey  questions,  responses  to 
computer-based  testing,  and  electrocardiographic  data.  Investigators  will  neither  collect 
nor  disclose  information  that  allows  linkage  of  the  data  to  a  specific  participant.  No 
personal  identifiers  will  be  included  in  the  data  sets  used  in  this  study.  All  participation  is 
voluntary.  In  order  to  avoid  any  possibility  for  inadvertently  collecting  medically  relevant 
information  from  ECG/EEG,  those  data  will  not  be  read  or  interpreted  by  humans  in  raw 
form;  only  machine-generated  summaries  of  heart  rate  and  heart  rate  variability  (standard 
deviation  of  heart  rate)  will  be  read  by  AFRL  personnel.  The  AFRL  personnel  doing  this 
analysis  are  not  medical  personnel  and  are  not  qualified  to  interpret  any  anomalous 
findings;  consequently,  no  attempt  will  be  made  to  read  raw  waveforms. 

Paperworkload  and  SA  scales  will  be  stored  in  a  locked  file  cabinet  within  a  room  in  B33, 
WPAFB  that  can  only  be  accessed  by  AFRL  staff;  electronic  data  will  be  stored  in 
removable  media  (secure  digital,  CD,  or  portable  hard  drive)  in  a  similar  locked  file 
cabinet.  When  reporting  the  results  of  this  study,  averages  and  standard  deviations  for 
each  of  the  primary  measures  as  well  as  statistically  significant  differences  among 
conditions  will  be  presented.  These  averages  would  contain  at  minimum  the  12 
participants  expected.  If  any  individual  exemplars  are  provided  to  illustrate  an  event,  only 
short  segments  of  data  without  coded  numeric  identifiers  will  be  shown.  Original  paper 
scales  will  never  be  copied  or  reproduced;  if  included  in  a  report  only  digitally  recoded 
data  will  be  included.  This  data  is  intended  to  be  provided  to  ASC  and  ANG  to  support 
equipment  and  deployment  decisions. 

9,  Risk  Analysis 

The  risk  is  minimal.  All  testing  will  be  done  using  standard  tests  that  are  non-invasive. 

The  probability  and  magnitude  of  harm  or  discomfort  anticipated  in  completing  the 
testing  is  not  greater  than  ordinarily  encountered  daily  life  events  or  during  the 
performance  of  completing  routine  physical  or  psychological  questionnaires.  To  mitigate 
any  potential  psychological  harm,  individual  results  will  never  be  disclosed.  This  is  also 
intended  to  substantiate  our  protection  against  adverse  effects  on  a  pilot’s  flight  status. 
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We  are  not  evaluating  individual  cockpit  performance.  The  use  of  disposable,  sterile 
ECG  /  EEG  leads  is  a  standard  procedure  conducted  in  an  outpatient  or  field 
environment  hundreds  of  times  each  day.  All  sorties  are  normal  training  sorties  with 
minimal  risk.  5 
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b.  Bedford  Workload  Scale 

c.  China  Lake  SA  Scale 

d.  Screenshot  of  IVA+,  mfr  information,  and  N-Back 

e.  Col.  McCue’s  letter  of  support 

f  Invitation  letter 

g.  Certification  letter  for  Vitaport  interference  testing  (EKG/EEG) 

h.  CITI  Certificate 
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Attachment:  Informed  Consent 

INFORMATION  PROTECTED  BY  THE  PRIVACY  ACT  OF 

1974 


Informed  Consent 
Document  For 

Usability  Testing  of  the  C-27J  HUD  and  predictors  of  Situation  Awareness  Research 

ANeuroergonomic  Quasi -Experiment:  Predictors  of  Situation  Awareness  and  Display  Usability  with  USAF 

Pilots  while  Perfonning  Comp lexTasks 
Usability  Testing  ofC-27J  HUD 

179  AW,  Mansfield,  Ohio 

71 1  HPW/RHCP,  WPAFB,  OH, 

Building  33 

A  Neuroergonomic  Quasi-Experiment:  Predictors  of  Situation  Awareness  and  Display 
Usability  with  USAF  Pilots  while  Performing  Complex  Tasks 

FWRH 

1,  Principal  Investigator 

Steven  Harbour,  Human  Factors  Engineer,  NOD  Doctoral  Candidate 
ASC/WLNJ,  937-255-8468 

Ste  ven .  h  a  rbou  r@wpaf  b .  af .  m  i  I 

2,  Associate  Investigator 

a.  Dr.  James  C.  Christensen,  Ph.D.,  DR-II,  Research  Psychologist,  NCU 
Dissertation  Chair  711  HPW/RHCP,  937-938-3603 
james.christensen@wpafb.af.mil 


1.  Nature  and  purpose:  You  have  been  offered  the  opportunity  to  participate  in  the 
“Usability  Testing  of  the  C-27J  HUD”  research  study  and  predictors  of  Situation 
Awareness.  Your  participation  will  occur  at  the  179  AW,  Mansfield,  Ohio. 

The  purpose  of  this  research  is  to  evaluate  the  impact  of  the  C-27J  HUD 
configuration  on  workload  and  situation  awareness  (SA),  and  to  discover  human 
antecedents  of  SA. 
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The  time  requirement  for  each  volunteer  subject  is  anticipated  to  be  an  additional 
30  minutes  added  to  the  normal  pre  and  post  sortie  activities.  This  project  will 
piggyback  on  normal  training  sorties  conducted  by  the  179  AW;  research 
personnel  will  not  direct,  schedule,  or  otherwise  interfere  in  training  missions.  A 
total  of  8  sorties  will  be  included  from  each  participant.  A  total  of  approximately  12 
subjects  will  be  enrolled  in  this  study. 


2.  Experimental  procedures:  If  you  decide  to  participate,  testing  will  involve  the  use 
of  survey  questionnaires  that  ask  about  workload  and  situation  awareness,  a 
computer-based  test  of  your  ability  to  divide  attention,  and  the  collection  of  heart 
activity  (electrocardiography-  ECG)  and  EEG  indicative  of  stress  and  workload  in 
flight.  To  record  the  electrical  activity  of  your  heart,  electrodes  will  be  attached  to 
your  collar  bone  and  chest,  and  EEG  to  your  scalp.  The  data  regarding  heart 
activity  will  not  be  read  by  a  human,  but  machine-coded  into  basic  data  about 
heart  rate,  and  standard  cognition  testing  (IVA+  and  N-back)  will  be  conducted  to 
be  explained  by  researcher  for  SA  antecedent  research.  The  Bedford  WL  and 
China  Lake  SA  surveys  will  be  handed  out  to  be  filled  in  at  the  end  of  the  sortie. 

The  personnel  conducting  this  study  will  not  examine  or  interpret  this  data  for  any 
medical  anomaly,  nor  are  they  qualified  to  do  so.  Individual  results  will  not  be 
shared  with  your  unit.  Participating  in  this  study  will  not  impact  your  flight  status. 
Under  that  protection,  we  encourage  you  to  be  completely  honest  and  provide  us 
with  as  much  feedback  as  possible.  In  this  way,  you  are  helping  us  to  generate 
sound,  quantitative  results  to  enable  the  best  decision-making  possible  regarding 
the  configuration  of  your  aircraft’s  systems  and  scientific  research  towards 
predicting  SA. 

3.  Discomfort  and  risks:  The  experimental  procedures  do  not  present  any 
unusual  or  risky  procedures  or  equipment.  However,  discomforts  may  consist  of 
mild  skin  irritation  where  the  ECG  /  EEG  electrodes  are  placed.  Potential  risks 
include  an  allergic  reaction  that  could  occur  at  the  site  of  electrode  placement; 
thus  participants  with  a  history  of  urticaria  (hives)  not  participate  in  the  study. 

4.  Precautions  for  female  subjects  or  subjects  wbo  are  or  may  become  pregnant 
during  tbe  course  of  this  study:  We  will  not  recruit  pregnant  women  for  this 
research  study. 

5.  Benefits:  You  are  not  expected  to  benefit  directly  from  participation  in  this  research 
study. 

6.  Compensation:  If  you  are  active  duty,  guard,  or  reserve  military  you  will 
receive  your  normal  duty  pay. 

7.  Alternatives:  Your  alternative  is  to  choose  not  to  participate  in  this  study. 
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Refusal  to  participate  will  involve  no  penalty  or  loss  of  benefits  to  which  you  are 
otherwise  entitled.  You  may  discontinue  participation  at  any  time  without 
penalty  or  loss  of  benefits  to  which  you  are  otherwise  entitled.  Notify  one  of  the 
investigators  of  this  study  to  discontinue. 

8.  Entitlements  and  confidentiality: 

a.  Records  of  your  participation  in  this  study  may  only  be  disclosed  according  to 
federal  law,  including  the  Federal  Privacy  Act,  5  U.S.C.  552a,  and  its 
implementing  regulations  and  the  Health  Insurance  Portability  and 
Accountability  Act  (HIPAA),  and  its  implementing  regulations,  when 
applicable,  and  the  Freedom  of  Information  Act,  5 

U.S.C.  Sec  552,  and  its  implementing  regulations  when  applicable.  Your  personal 
information  will  be  stored  in  a  locked  cabinet  in  an  office  that  is  locked  when  not 
occupied.  Electronic  files  containing  your  personal  information  will  be  password 
protected  and  stored  only  on  a  secure  server.  It  is  intended  that  the  only  people 
having  access  to  your  information  will  be  the  researchers  named  above  and  this 
study’s  Medical  Monitor  or  Consultant,  the  AFRL  Wright  Site  IRB,  the  Air  Force 
Surgeon  General’s  Research  Compliance  office,  the  Director  of  Defense  Research 
and  Engineering  office  or  any  other  IRB  involved  in  the  review  and  approval  of  this 
protocol.  When  no  longer  needed  for  research  purposes  your  information  will  be 
destroyed  in  a  secure  manner  (shredding).  Complete  confidentiality  cannot  be 
promised,  in  particular  for  military  personnel,  whose  health  or  fitness  for  duty 
information  may  be  required  to  be  reported  to  appropriate  medical  or  command 
authorities.  If  such  information  is  to  be  reported,  you  will  be  informed  of  what  is  being 
reported  and  the  reason  for  the  report. 

b.  Your  entitlements  to  medical  and  dental  care  and/or  compensation  in  the  event 
of  injury  are  governed  by  federal  laws  and  regulations,  and  that  if  you  desire 
further  information  you  may  contact  the  base  legal  office  (ASC/JA,  257-6143 
for  Wright-Patterson  AFB).  In  the  event  of  a  research  related  injury,  you  may 
contact  the  AFRL  IRB  office  at  937-656-  5689  or  afrl.heh.dl.irb@wpafb.af.mil. 

c.  If  an  unanticipated  event  (medical  misadventure)  occurs  during  your 
participation  in  this  study,  you  will  be  informed.  If  you  are  not  competent  at 
the  time  to  understand  the  nature  of  the  event,  such  information  will  be 
brought  to  the  attention  of  your  next  of  kin  or  other  listed  emergency  contact. 

The  decision  to  participate  in  this  research  is  completely  voluntary  on  your  part.  No 
one  may  coerce  or  intimidate  you  into  participating  in  this  program.  You  are 
participating  because  you  want  to.  Steven  D.  Harbour,  or  an  associate,  has 
adequately  answered  any  and  all  questions  you  have  about  this  study,  your 
participation,  and  the  procedures  involved.  Steven  D.  Harbour  can  be  reached  at 
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(937)  255-8748.  Steven  D.  Harbour  or  an  associate  will  be  available  to  answer  any 
questions  concerning  procedures  throughout  this  study.  If  significant  new  findings 
develop  during  the  course  of  this  research,  which  may  relate  to  your  decision  to 
continue  participation,  you  will  be  informed.  Refusal  to  participate  will  involve  no 
penalty  or  loss  of  benefits  to  which  you  are  otherwise  entitled.  You  may  discontinue 
participation  at  any  time  without  penalty  or  loss  of  benefits  to  which  you  are  otherwise 
entitled.  Notify  one  of  the  investigators  of  this  study  to  discontinue.  The  investigator 
or  medical  monitor  of  this  study  may  terminate  your  participation  in  this  study  if  she  or 
he  feels  this  to  be  in  your  best  interest.  If  you  have  any  questions  or  concerns  about 
your  participation  in  this  study  or  your  rights  as  a  research  subject,  please  contact  Col 
William  P.  Butler;  937-656-5437  or  william.butler3@wpafb.af.mil. 

e.  No  personally  identifiable  information  will  be  obtained  for  this  study,  unless  you 
consent  below  to  photography  and/or  videotaping.  Such  recordings,  if  made, 
will  not  show  your  face  or  name  tape. 

f.  Your  participation  in  this  study  may  be  photographed,  filmed  or 
audio/videotaped.  The  purpose  of  these  recordings  is  for  training  and  data 
collection  purposes.  Any  release  of  records  of  your  participation  in  this  study 
may  only  be  disclosed  according  to  federal  law,  including  the  Federal  Privacy 
Act,  55  U.S.C.  552a,  and  its  implementing  regulations.  This  means  personal 
information  will  not  be  released  to  unauthorized  source  without  your 
permission.  These  recording  may  be  used  for  presentation  or  publication,  with 
your  signed  permission.  They  will  be  stored  in  a  locked  cabinet  in  a  room  that 
is  locked  when  not  occupied.  Only  the  investigators  of  this  study  will  have 
access  to  these  media.  They  will  be  maintained  for  5  years. 

YOU  ARE  MAKING  A  DECISION  WHETHER  OR  NOT  TO  PARTICIPATE.  YOUR 

SIGNATURE  INDICATES  THAT  YOU  HAVE  DECIDED  TO  PARTICIPATE 

HAVING  READ  THE  INFORMATION  PROVIDED  ABOVE. 


Volunteer  Signature _ Date. 

Volunteer  Name  (printed) _ 

Advising  Investigator  Signature _ Date _ 

Investigator  Name  (printed) _ 

Witness  Signature _ Date _ 

Witness  Name  (printed) _ 
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We  may  wish  to  present  some  of  the  video/audio  recordings  or  photographs  from  this 
study  as  part  of  reporting  the  outcome  to  Air  Force  leadership,  and/or  in  sharing 
scientific  results  with  the  research  community  in  an  academic  conference  setting.  If 
you  consent  to  the  use  of  your  image  for  publication  or  presentation  in  such  settings, 
please  sign  below.  Choosing  not  to  sign  will  not  impact  your  participation  in  the 
study. 

Volunteer  Signature _ Date _ 

Privacy  Act  Statement 


Authority:  We  are  requesting  disclosure  of  personal  information. .  Researchers  are  authorized  to  collect  personal 
information  on  research  subjects  under  The  Privacy  Act-5  USC  552a,  10  USC  55, 10  USC  8013,  32  CFR219, 45 
CFRPart  46,  andEO  9397,  November  1943. 

Piirnnse:  It  is  possible  that  latent  risks  or  injuries  inherent  in  this  experiment  will  not  be  discovered  until  some 
time  in  the  future.  The  purpose  of  collecting  this  information  is  to  aid  researchers  in  locating  you  at  a  future 
date  if  further  disclosures  are  appropriate. 

Routine  Uses:  Information  maybe  furnished  to  F  ederal.  State  and  local  agencies  for  any  uses  published  by  the  Air 
Force  in  the  Federal  Register,  52  FR 16431,  to  include,  furtherance  ofthe  research  involved  with  this  study  and  to 
provide  medical  care. 

Disclosure:  Disclosure  ofthe  requested  information  is  voluntary.  No  adverse  action  whatsoever  will  be  taken 
against  you,  and  no  privilege  will  be  denied  you  based  on  the  fact  you  do  not  disclose  this  information. 

However,  your  participation  in  this  study  may  be  impacted  by  a  refusal  to  provide  this  information. 
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